-- Logs begin at Fri 2022-10-14 01:46:37 UTC, end at Fri 2022-10-14 02:43:34 UTC. -- Oct 14 02:10:48.615740 dv-cinder-845069-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 14 02:10:48.623809 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 14 02:10:48.626586 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Oct 14 02:10:48 2022] *** Oct 14 02:10:48.626721 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 14 02:10:48.626827 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 14 02:10:48.626937 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: nodename: dv-cinder-845069-4-iscsi Oct 14 02:10:48.627022 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: machine: x86_64 Oct 14 02:10:48.627161 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: clock source: unix Oct 14 02:10:48.627262 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: pcre jit disabled Oct 14 02:10:48.627361 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: detected number of CPU cores: 4 Oct 14 02:10:48.627458 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: current working directory: / Oct 14 02:10:48.627555 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: detected binary path: /usr/bin/uwsgi-core Oct 14 02:10:48.627669 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: your processes number limit is 47680 Oct 14 02:10:48.627770 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: your memory page size is 4096 bytes Oct 14 02:10:48.627870 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: detected max file descriptor number: 2048 Oct 14 02:10:48.627971 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: lock engine: pthread robust mutexes Oct 14 02:10:48.628228 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: thunder lock: enabled Oct 14 02:10:48.628509 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 14 02:10:48.628688 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 14 02:10:48.668021 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: Python main interpreter initialized at 0x564af55ea9c0 Oct 14 02:10:48.668021 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: python threads support enabled Oct 14 02:10:48.668021 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: your server socket listen backlog is limited to 100 connections Oct 14 02:10:48.668021 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: your mercy for graceful operations on workers is 90 seconds Oct 14 02:10:48.668362 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: mapped 403077 bytes (393 KB) for 2 cores Oct 14 02:10:48.668450 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: *** Operational MODE: preforking *** Oct 14 02:10:48.668698 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: *** uWSGI is running in multiple interpreter mode *** Oct 14 02:10:48.668698 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: spawned uWSGI master process (pid: 81349) Oct 14 02:10:48.668897 dv-cinder-845069-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 14 02:10:48.669430 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: spawned uWSGI worker 1 (pid: 81361, cores: 1) Oct 14 02:10:48.669761 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: spawned uWSGI worker 2 (pid: 81362, cores: 1) Oct 14 02:10:48.669914 dv-cinder-845069-4-iscsi devstack@keystone.service[81349]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 14 02:10:52.945216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 14 02:10:52.945216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: from cryptography.utils import int_from_bytes Oct 14 02:10:52.946004 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 14 02:10:52.946004 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: from cryptography.utils import int_from_bytes Oct 14 02:10:52.946415 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 14 02:10:52.946415 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: from cryptography.utils import int_from_bytes Oct 14 02:10:52.946519 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 14 02:10:52.946519 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: from cryptography.utils import int_from_bytes Oct 14 02:10:52.995799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 14 02:10:52.995949 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 14 02:10:52.996105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] command line args: [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 14 02:10:52.996194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 14 02:10:52.996358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 14 02:10:52.996441 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 14 02:10:52.996543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ================================================================================ {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 14 02:10:52.996626 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] command line args: [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 14 02:10:52.996798 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] admin_token = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.996881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 14 02:10:52.996978 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ================================================================================ {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 14 02:10:52.997053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] config_dir = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.997225 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.997305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] admin_token = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.997410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] config_source = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.997489 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] config_dir = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.997591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] debug = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.997672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.997791 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.998048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] config_source = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.998048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] debug = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.998160 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.998216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.998391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.998474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.998573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.998648 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.998814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.998898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.999045 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] list_limit = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.999153 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.999247 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] log_config_append = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.999318 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] list_limit = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.999670 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.999753 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] log_config_append = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.999852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] log_dir = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:52.999928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.000025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] log_file = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.000128 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] log_dir = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.000221 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] log_file = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.000322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.000468 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.000551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.000671 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.000748 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.000916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.001042 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.001122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.001242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.001316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.001436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.001519 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.001588 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.001734 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.001815 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.001970 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.002094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.002204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.002279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.002436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.002508 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.002666 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.002750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.002904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.003021 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.003171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.003254 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.003420 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.003494 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.003699 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.003927 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.004112 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] publish_errors = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.004182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.004348 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.004424 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] publish_errors = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.004581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.004658 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.004760 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.004835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.004982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.005060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.005232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.005314 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.005466 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] standard_threads = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.005557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.005729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.005811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] standard_threads = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.005972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.006171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.006255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.006324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.006397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] use_journal = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.006466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.006583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] use_json = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.006660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] use_journal = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.006734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] use_stderr = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.006808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] use_json = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.006881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] use_syslog = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.006955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] use_stderr = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.007030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.007149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] use_syslog = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.007373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.007458 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:10:53.007612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.007686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.007863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.007943 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.008102 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.008179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.008352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.008427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.008590 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.008673 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.008830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.008904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.009067 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.009140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.009312 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.009392 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.009556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.009635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.009805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.009880 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.010037 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.010118 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.010281 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.010366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.010519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.010637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.010758 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.010833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.011003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.011107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.011261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.011344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.011517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.011599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.011771 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.011846 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.012016 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.012090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.012256 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.012333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.012493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.012568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.012750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.012832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.013003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.013078 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.013253 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.013327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.013486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.013569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.013717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.013792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.013948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.014024 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.014180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.014261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.014420 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.014500 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.014664 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.014739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.014894 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.014976 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.015149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.015229 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.015388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.015461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.015635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.015718 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.016723 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.016879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.016879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.017117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.017117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.017361 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.017499 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.017629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.017759 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.017887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.018011 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.018141 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.019862 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.020012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.020141 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.020219 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.020277 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.020412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.020491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.020567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.020636 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.020708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.020777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.020852 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.020969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.021041 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.021108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.021714 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.021785 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.021956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.022031 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.022099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.022165 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.022231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.022330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.022432 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.022526 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.022618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.022708 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.022798 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.022889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.022993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.023122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.023222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.023353 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.023459 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.023543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.023608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.023670 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.023731 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.023797 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.023859 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.023928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.023997 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.024066 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.024136 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.024204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.024267 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.024340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.024416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.024497 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.024561 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.024628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.024692 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.024758 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.024820 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.024897 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.024966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025221 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025298 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025496 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025570 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025712 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025781 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025843 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025909 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.025971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.026040 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.026111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.026184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.026256 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.026326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.026388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.026455 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.026532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.026670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.026771 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.026920 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.027014 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.027183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.027270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.027422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.027497 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.027662 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.027738 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.027899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.027978 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.028137 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.028217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.028401 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.028476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.028639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.028720 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.028877 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.028952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.029117 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.029190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.029352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.029502 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.029575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.029643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.029809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.029885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.030038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.030124 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.030267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.030343 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.030495 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.030569 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.030728 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.030803 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.030957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.031032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.031217 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.031291 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.031451 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.031525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.031695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.031772 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.031930 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.032003 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.032179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.032254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.032418 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.032494 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.032663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.032747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.032896 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] auth.external = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.033002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.033128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.033204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.033385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] auth.external = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.033460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.033617 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.033694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.033856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] auth.password = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.033939 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.034085 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] auth.token = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.034162 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.034321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.034396 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] auth.password = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.034555 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.034637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] auth.token = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.034801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.034877 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.035030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.035123 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.035300 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.035376 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.035532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.035608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.035764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.035845 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.035999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] credential.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.036075 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.036239 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.036315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.036477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] credential.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.036552 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.036723 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.037459 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.037550 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.037631 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.037762 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.037871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.037995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.038072 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.038241 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.038317 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.038478 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.038554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.038716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.038791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.038968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.039044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.039232 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.039312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.039489 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.039564 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.039757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.039840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.039999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.040076 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.040254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.040332 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.040490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.040574 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.040727 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.040803 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.040971 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.041046 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.041211 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.041288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.041438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.041519 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.041685 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.041769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.041916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.041997 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.042165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.042242 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.042398 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.042484 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.042652 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.042728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.042887 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.042962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.043159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.043291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.043396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.043478 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.043643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.043718 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.043882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.044030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.044104 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.044171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.044338 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.044415 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.044585 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.044669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.044816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.044891 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.045050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.045126 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.045284 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.045366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.045523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.045599 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.045751 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.045825 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.045981 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.046065 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.046211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.046287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.046454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.046529 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.046689 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.046772 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.046932 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.047007 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.047205 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.047281 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.047448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.047535 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.047706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.047782 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.047939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.048013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.048175 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.048262 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.048405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.048481 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.048639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.048716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.048869 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.048952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.049110 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.049184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.049345 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.049420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.049585 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.049668 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.049829 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.049905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.050067 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.050141 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.050316 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.050400 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.050552 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.050627 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.050785 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.050859 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.051030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.051138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.051294 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.051369 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.051520 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.051595 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.051758 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.051835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.051991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.052067 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.052226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.052301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.052483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.052564 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.052722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.052806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.052960 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.053095 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.053204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.053449 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.053525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.053703 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.053803 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.053956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.054061 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.054190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.054314 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.054422 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.054550 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.054673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.054786 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.054921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.055038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.055177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.055316 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.055412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.055556 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.055652 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.055795 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.055879 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.056035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.056110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.056269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.056346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.056512 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.056588 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.056775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.056851 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.057010 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.057087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.057261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.057337 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.057501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.057575 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.057732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.057810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.057968 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.058044 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.058196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.058271 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.058430 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.058505 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.058660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.058736 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.058895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.058969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.059149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.059231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.059395 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.059470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.059640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.059724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.059883 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.059959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.060114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.060190 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.060358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.060439 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.060591 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.060668 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.060824 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.060898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.061057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.061133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.061290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.061366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.061527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.061602 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.061763 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.061839 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.062002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.062078 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.062252 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.062327 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.062495 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.062576 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.062721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.062799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.062955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.063030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.063209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.063284 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.063444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.063522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.063690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.063765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.063926 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.064001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.064156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.064232 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.064394 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.064470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.064626 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.064700 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.064864 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.064941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.065108 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.065183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.065340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.065415 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.065588 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.065665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.065822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.065896 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.066058 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.066132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.066303 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.066380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.066537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.066612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.066775 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.066926 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.066999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] role.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.067087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.067266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] role.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.067342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.067495 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.067583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] role.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.067742 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.067819 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] role.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.067972 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.068047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.068247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.068375 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.068629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.068704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.068878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.068954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.069126 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.069209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.069374 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.069450 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.069622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.069698 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.069853 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.069974 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.070098 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.070175 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.070333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.070407 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.070584 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.070668 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.070827 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.070902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.071088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.071246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.071330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.071483 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.071727 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.071804 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.071968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.072043 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.072215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.072299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.072463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.072538 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.072695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.072772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.072947 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.073024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.073183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.073259 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.073423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.073497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.073653 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.073730 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.073883 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.073959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.074113 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.074188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.074352 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.074428 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.074585 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.074661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.074836 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.074911 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.075084 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.075165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.075328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.075405 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.075558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.075650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.075795 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.075872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.076026 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.076100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.076276 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.076351 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.076501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.076576 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.076751 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.076825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.076990 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.077065 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.077225 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.077301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.077469 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.077545 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.077707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.077782 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.077952 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.078028 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.078186 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.078262 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.078423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.078499 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.078657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.078745 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.078896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.078971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.079157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.079238 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.079409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.079487 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.079647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.079732 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.079901 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.079984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.080146 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.080223 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.connection = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.080386 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.080469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.080631 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.080708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.connection = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.080867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.080943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.081107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.081192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.081341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.081424 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.081575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.081651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.081807 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.081891 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.082042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.082118 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.082278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.082358 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.082509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.082592 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.082750 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.082822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.082982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.083056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.083248 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.083338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.083482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.083559 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.084269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.084350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.084571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.084766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.084847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.084920 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.085041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.085155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.085231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 14 02:10:53.085396 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.085634 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:10:53.085805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 14 02:10:53.114496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.114725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.114824 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.115500 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.115662 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.115899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.123883 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.124293 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.153030 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.153228 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.153317 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.153458 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.207422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.207579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.257348 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.257470 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.257582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.257665 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.416068 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.416294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.416509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.416730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.416953 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.417059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.417171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.417252 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.417438 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.417666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.417896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.418116 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.440022 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.440253 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.440372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.440480 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.440611 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.440692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.440843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.440924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.441069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.441158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.441289 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.441367 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.441506 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.441720 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.452987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.453347 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.466271 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.466506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.466587 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.466733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.466817 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.466961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.467091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.467212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.467387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.467479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.467631 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.467709 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.467898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.468120 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:10:53.484234 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.484444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 14 02:10:53.484791 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.484993 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 14 02:10:53.485269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.485533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.485797 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.486056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.487402 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.487642 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 14 02:10:53.487992 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.488208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 14 02:10:53.488503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.488805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.489163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.489466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.496226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.496507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.496782 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.497065 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.500330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.500615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.500893 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.501182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.504289 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.507938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.508183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.508468 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.508699 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.508778 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.509025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.509289 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.509564 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.509841 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.510112 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.510386 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.510655 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.511613 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.511903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.512190 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.512477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.512769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.513041 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.513320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.513599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.513872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.514150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.514422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.532962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.534421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.534705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.537238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.538777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.539085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.539373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.539660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.539946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.543741 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.544034 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.544324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.546196 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.546487 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.550624 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.550917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.552062 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.552866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.553238 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.553597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.553878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.554159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.554435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.554709 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.557161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.557458 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.557756 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.558048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.558331 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.558613 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.558891 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.559191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.570727 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.571006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.574839 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.575044 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.575354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.578063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.578363 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.579201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.582440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.582519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.582744 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.585701 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.587100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.588888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.589193 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.589484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.589773 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.590055 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.590310 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.590433 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.593549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.593878 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.594190 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.594484 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.594772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.595101 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.606062 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.606352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.606626 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.606897 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.611020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.611337 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.611609 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.611876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.613681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.615753 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.617287 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.617568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.619474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.620313 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.620597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.620888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.621011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.621186 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.621582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.622544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.622826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.625834 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.626121 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.626409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.626716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.627150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.632933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.633222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.633510 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.633803 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.634193 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.634493 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.634778 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.638665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.638958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.639276 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.639569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.639970 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.640270 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.640569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.650478 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.653370 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.656254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.656509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.657844 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.659359 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.659485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.659645 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.660049 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.662909 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.664469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.665983 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.666270 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.666679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.666927 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.668415 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.671296 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.671693 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.671985 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.672267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.673119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.674587 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.677557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.677994 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.678285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.678573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.680220 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.680513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.680798 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.686408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.686725 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.687035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.687638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.687922 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.688204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.688482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.688756 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.689041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.689315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.689602 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.689881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.690166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.690442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.690719 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.691010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.691310 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.694055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:10:53.694505 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.694796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.695091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.695381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.695661 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.695940 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.696230 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.696518 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.696810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.697098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.697365 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.697636 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.697910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:10:53.726477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564af55ea9c0 pid: 81361 (default app) Oct 14 02:10:53.729911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162f968-0cbd-46cb-b4ad-61837633c416 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:10:53.730201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162f968-0cbd-46cb-b4ad-61837633c416 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:10:53.730474 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162f968-0cbd-46cb-b4ad-61837633c416 None None] PATH_INFO: `/v3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:10:53.732862 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564af55ea9c0 pid: 81362 (default app) Oct 14 02:10:53.734662 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Fri Oct 14 02:10:53 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:00.458816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fe45b-88db-4d5b-84fc-1ed5fea77146 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:00.459385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fe45b-88db-4d5b-84fc-1ed5fea77146 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:00.459385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fe45b-88db-4d5b-84fc-1ed5fea77146 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:00.463749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:00.469260 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981c21b-c4da-40bf-87ce-600b0f40828b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:00.469820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981c21b-c4da-40bf-87ce-600b0f40828b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:00.469820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981c21b-c4da-40bf-87ce-600b0f40828b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:00.542599 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: INFO dbcounter [None req-c981c21b-c4da-40bf-87ce-600b0f40828b None None] Registered counter for database keystone Oct 14 02:11:00.562171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG oslo_db.sqlalchemy.engines [None req-c981c21b-c4da-40bf-87ce-600b0f40828b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81361) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 14 02:11:00.569595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writer thread running {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 14 02:11:00.752458 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG passlib.handlers.bcrypt [None req-c981c21b-c4da-40bf-87ce-600b0f40828b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81361) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 14 02:11:00.752852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG passlib.handlers.bcrypt [None req-c981c21b-c4da-40bf-87ce-600b0f40828b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81361) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 14 02:11:00.812259 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c981c21b-c4da-40bf-87ce-600b0f40828b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:00.813835 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c981c21b-c4da-40bf-87ce-600b0f40828b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:00.943656 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 14 02:11:01.421111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fe45b-88db-4d5b-84fc-1ed5fea77146 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:01.496828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: INFO dbcounter [None req-0d4fe45b-88db-4d5b-84fc-1ed5fea77146 None None] Registered counter for database keystone Oct 14 02:11:01.518509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG oslo_db.sqlalchemy.engines [None req-0d4fe45b-88db-4d5b-84fc-1ed5fea77146 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81362) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 14 02:11:01.524849 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writer thread running {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 14 02:11:01.642501 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fe45b-88db-4d5b-84fc-1ed5fea77146 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:01.646845 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49cd57-3f94-451c-8e24-cdbfc7531262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:01.647498 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49cd57-3f94-451c-8e24-cdbfc7531262 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:01.647778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49cd57-3f94-451c-8e24-cdbfc7531262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:01.648051 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49cd57-3f94-451c-8e24-cdbfc7531262 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:01.656148 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49cd57-3f94-451c-8e24-cdbfc7531262 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:01.709015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cf49cd57-3f94-451c-8e24-cdbfc7531262 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 14 02:11:01.709302 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49cd57-3f94-451c-8e24-cdbfc7531262 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:01.718498 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Fri Oct 14 02:11:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:11:01.725197 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981c21b-c4da-40bf-87ce-600b0f40828b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:01.742713 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981c21b-c4da-40bf-87ce-600b0f40828b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:01.747151 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c6d78-5508-4a19-8f76-2714e64e7635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:01.748261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c6d78-5508-4a19-8f76-2714e64e7635 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:01.748547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c6d78-5508-4a19-8f76-2714e64e7635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:01.748820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c6d78-5508-4a19-8f76-2714e64e7635 admin admin] PATH_INFO: `/v3/projects/c1e7392d098a4e219827c4ae7f8117e2` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:01.750654 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c6d78-5508-4a19-8f76-2714e64e7635 admin admin] RBAC: Authorizing `identity:get_project(project_id=c1e7392d098a4e219827c4ae7f8117e2)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:01.804297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: WARNING keystone.common.rbac_enforcer.enforcer [None req-120c6d78-5508-4a19-8f76-2714e64e7635 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 14 02:11:01.804652 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c6d78-5508-4a19-8f76-2714e64e7635 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:01.809234 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Fri Oct 14 02:11:01 2022] GET /identity/v3/projects/c1e7392d098a4e219827c4ae7f8117e2 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:03.920058 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c0927-790e-4e1e-a923-581a937bcbd3 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:03.920618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c0927-790e-4e1e-a923-581a937bcbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:03.920618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c0927-790e-4e1e-a923-581a937bcbd3 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:03.924785 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:11:03.930750 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823780c1-91fe-4b82-9510-39743a77e186 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:03.931197 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823780c1-91fe-4b82-9510-39743a77e186 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:03.931341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823780c1-91fe-4b82-9510-39743a77e186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:03.950797 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-823780c1-91fe-4b82-9510-39743a77e186 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:03.952343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-823780c1-91fe-4b82-9510-39743a77e186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:03.960087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 14 02:11:04.226909 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c0927-790e-4e1e-a923-581a937bcbd3 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:04.238742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c0927-790e-4e1e-a923-581a937bcbd3 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:04.242882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab909f7-3c7f-4cd6-8056-49cfd920ac00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:04.243512 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab909f7-3c7f-4cd6-8056-49cfd920ac00 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:04.243791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab909f7-3c7f-4cd6-8056-49cfd920ac00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:04.244060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab909f7-3c7f-4cd6-8056-49cfd920ac00 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:04.251752 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab909f7-3c7f-4cd6-8056-49cfd920ac00 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:04.253513 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab909f7-3c7f-4cd6-8056-49cfd920ac00 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:04.261980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Fri Oct 14 02:11:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:11:04.267320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823780c1-91fe-4b82-9510-39743a77e186 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:04.276836 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823780c1-91fe-4b82-9510-39743a77e186 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:04.281137 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa0c98-592c-4cd0-91dc-471e8505ecaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:04.281759 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa0c98-592c-4cd0-91dc-471e8505ecaa admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:04.282038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa0c98-592c-4cd0-91dc-471e8505ecaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:04.282312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa0c98-592c-4cd0-91dc-471e8505ecaa admin admin] PATH_INFO: `/v3/users/6f436f18778d45bb9d581131675fcbb4` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:04.283636 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa0c98-592c-4cd0-91dc-471e8505ecaa admin admin] RBAC: Authorizing `identity:get_user(user_id=6f436f18778d45bb9d581131675fcbb4)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:04.285456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa0c98-592c-4cd0-91dc-471e8505ecaa admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:04.287587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Fri Oct 14 02:11:04 2022] GET /identity/v3/users/6f436f18778d45bb9d581131675fcbb4 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:06.458876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae4e5d-2f24-4f9c-af37-8307085ca5f7 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:06.459451 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae4e5d-2f24-4f9c-af37-8307085ca5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:06.459532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae4e5d-2f24-4f9c-af37-8307085ca5f7 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:06.460833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:06.465971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f739ba2-d9db-4821-8ea6-cfaeaa991a3c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:06.466377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f739ba2-d9db-4821-8ea6-cfaeaa991a3c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:06.466522 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f739ba2-d9db-4821-8ea6-cfaeaa991a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:06.484720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5f739ba2-d9db-4821-8ea6-cfaeaa991a3c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:06.485900 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5f739ba2-d9db-4821-8ea6-cfaeaa991a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:06.525742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac93c87-bb94-420e-8f71-d11df0e3ca57 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:06.526023 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac93c87-bb94-420e-8f71-d11df0e3ca57 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:06.526293 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac93c87-bb94-420e-8f71-d11df0e3ca57 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:06.527624 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 6/11] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:11:06.532742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c52aa-f64a-4982-933f-336c7eada4e3 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:06.533023 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c52aa-f64a-4982-933f-336c7eada4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:06.533298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c52aa-f64a-4982-933f-336c7eada4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:06.567672 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 6/12] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:06.574745 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG passlib.handlers.bcrypt [None req-7c3c52aa-f64a-4982-933f-336c7eada4e3 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81362) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 14 02:11:06.575158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG passlib.handlers.bcrypt [None req-7c3c52aa-f64a-4982-933f-336c7eada4e3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81362) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 14 02:11:06.625150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7c3c52aa-f64a-4982-933f-336c7eada4e3 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:06.626688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7c3c52aa-f64a-4982-933f-336c7eada4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:06.726006 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 7/13] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:11:06.833951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f739ba2-d9db-4821-8ea6-cfaeaa991a3c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:06.859018 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:06.939375 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f739ba2-d9db-4821-8ea6-cfaeaa991a3c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:06.973242 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd243fa-774a-4339-8b26-a2c6edbd0a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:06.973945 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd243fa-774a-4339-8b26-a2c6edbd0a89 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:06.974268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd243fa-774a-4339-8b26-a2c6edbd0a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:06.974550 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd243fa-774a-4339-8b26-a2c6edbd0a89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:06.985599 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd243fa-774a-4339-8b26-a2c6edbd0a89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:06.991329 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c52aa-f64a-4982-933f-336c7eada4e3 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-8cd243fa-774a-4339-8b26-a2c6edbd0a89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:06.995383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:06.996235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:06.997047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:06.997047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:06.997047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:06.997047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:06.997047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:06.997047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:06.997047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:06.997047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:06.997047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:06.997047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:06.997047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:11:06.997047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:06.997047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 7/14] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:11:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:11:07.002464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd243fa-774a-4339-8b26-a2c6edbd0a89 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:07.082111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:07.092355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c52aa-f64a-4982-933f-336c7eada4e3 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:07.094601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:07.099895 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd243fa-774a-4339-8b26-a2c6edbd0a89 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:07.132719 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b226a35-2317-41d7-a10f-c094ab0f3f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:07.133591 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b226a35-2317-41d7-a10f-c094ab0f3f0a None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:07.133876 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbdf28-6faf-4d41-a250-2588778bf42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:07.133992 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b226a35-2317-41d7-a10f-c094ab0f3f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:07.134148 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b226a35-2317-41d7-a10f-c094ab0f3f0a None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:07.134488 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbdf28-6faf-4d41-a250-2588778bf42e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:07.134768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbdf28-6faf-4d41-a250-2588778bf42e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:07.135045 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbdf28-6faf-4d41-a250-2588778bf42e None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:07.135160 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b226a35-2317-41d7-a10f-c094ab0f3f0a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:07.135738 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbdf28-6faf-4d41-a250-2588778bf42e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:07.136038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbdf28-6faf-4d41-a250-2588778bf42e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:07.136573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b226a35-2317-41d7-a10f-c094ab0f3f0a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:07.137401 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbdf28-6faf-4d41-a250-2588778bf42e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:07.147879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.sql.core [None req-8b226a35-2317-41d7-a10f-c094ab0f3f0a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 02:11:07.147879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 02:11:07.147879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [parameters: {'id': '8a02e91284d0493181b8114f5736a829', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 14 02:11:07.147879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81362) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 02:11:07.149352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 8/15] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:07.154826 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbdf28-6faf-4d41-a250-2588778bf42e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:07.166523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-8b226a35-2317-41d7-a10f-c094ab0f3f0a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 14 02:11:07.166523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:07.166523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:11:07.166523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:11:07.166523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:11:07.166523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:11:07.166523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:11:07.166523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:11:07.166523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:11:07.166523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:11:07.166523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:11:07.166523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:11:07.166523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:11:07.166523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result.read() Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 14 02:11:07.167351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application yield resource Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise self.value Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application session.commit() Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 02:11:07.171639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.session.flush() Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application compat.raise_( Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 02:11:07.172437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 02:11:07.173295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result.read() Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:11:07.174008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [parameters: {'id': '8a02e91284d0493181b8114f5736a829', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 14 02:11:07.174826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:07.175583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:07.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 14 02:11:07.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 14 02:11:07.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:07.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:07.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 14 02:11:07.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 14 02:11:07.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:07.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:07.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 14 02:11:07.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 14 02:11:07.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 14 02:11:07.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:07.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 8/16] 10.222.0.18 () {60 vars in 1137 bytes} [Fri Oct 14 02:11:06 2022] POST /identity/v3/domains => generated 190 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:11:07.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b226a35-2317-41d7-a10f-c094ab0f3f0a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:07.258712 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbdf28-6faf-4d41-a250-2588778bf42e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:07.276716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b226a35-2317-41d7-a10f-c094ab0f3f0a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:07.290855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58da89-06d7-4413-bbad-7b292392fbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:07.291521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58da89-06d7-4413-bbad-7b292392fbec None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:07.291805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58da89-06d7-4413-bbad-7b292392fbec None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:07.292103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58da89-06d7-4413-bbad-7b292392fbec None admin] PATH_INFO: `/v3/users/6f436f18778d45bb9d581131675fcbb4` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:07.293207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58da89-06d7-4413-bbad-7b292392fbec None admin] RBAC: Authorizing `identity:get_user(user_id=6f436f18778d45bb9d581131675fcbb4)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:07.294652 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58da89-06d7-4413-bbad-7b292392fbec None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:07.296240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 9/17] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:07 2022] GET /identity/v3/users/6f436f18778d45bb9d581131675fcbb4 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:07.301561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58da89-06d7-4413-bbad-7b292392fbec None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:07.309541 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd91b59-3df0-469c-a251-8af01d58a115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:07.310429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd91b59-3df0-469c-a251-8af01d58a115 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:07.310717 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd91b59-3df0-469c-a251-8af01d58a115 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:07.310993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd91b59-3df0-469c-a251-8af01d58a115 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:07.316417 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:07.328783 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd91b59-3df0-469c-a251-8af01d58a115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:07.330270 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd91b59-3df0-469c-a251-8af01d58a115 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-7bd91b59-3df0-469c-a251-8af01d58a115 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:11:07.346801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:07.347688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:11:07.348456 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:11:07.348456 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:11:07.348456 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:07.348456 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 9/18] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:11:07.353907 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd91b59-3df0-469c-a251-8af01d58a115 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:07.377137 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:07.401315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58da89-06d7-4413-bbad-7b292392fbec None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:07.434777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85376264-559e-4c16-a003-7c3aad2a99cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:07.435434 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85376264-559e-4c16-a003-7c3aad2a99cd None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:07.435716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85376264-559e-4c16-a003-7c3aad2a99cd None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:07.435988 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85376264-559e-4c16-a003-7c3aad2a99cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:07.437099 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85376264-559e-4c16-a003-7c3aad2a99cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:07.438546 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85376264-559e-4c16-a003-7c3aad2a99cd None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:07.440127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 10/19] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:07.446335 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85376264-559e-4c16-a003-7c3aad2a99cd None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:07.454373 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd91b59-3df0-469c-a251-8af01d58a115 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:07.490062 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691de572-13a0-4608-b23e-35a3d78f43bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:07.490682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691de572-13a0-4608-b23e-35a3d78f43bc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:07.490929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691de572-13a0-4608-b23e-35a3d78f43bc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:07.491223 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691de572-13a0-4608-b23e-35a3d78f43bc None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:07.491862 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691de572-13a0-4608-b23e-35a3d78f43bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:07.492161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691de572-13a0-4608-b23e-35a3d78f43bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:07.493528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691de572-13a0-4608-b23e-35a3d78f43bc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:07.510793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 10/20] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:07.527474 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:07.548225 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85376264-559e-4c16-a003-7c3aad2a99cd None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:07.584164 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9894a37-e682-4581-ae87-dc81937c8c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:07.584849 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9894a37-e682-4581-ae87-dc81937c8c5d None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:07.585129 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9894a37-e682-4581-ae87-dc81937c8c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:07.585407 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9894a37-e682-4581-ae87-dc81937c8c5d None admin] PATH_INFO: `/v3/domains/default/users/6f436f18778d45bb9d581131675fcbb4/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:07.587429 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9894a37-e682-4581-ae87-dc81937c8c5d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6f436f18778d45bb9d581131675fcbb4, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:07.588902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9894a37-e682-4581-ae87-dc81937c8c5d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:07.705641 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 11/21] 10.222.0.18 () {58 vars in 1361 bytes} [Fri Oct 14 02:11:07 2022] PUT /identity/v3/domains/default/users/6f436f18778d45bb9d581131675fcbb4/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:11:10.129777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e1ec9-f396-47dd-81f2-30faf6f40040 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:10.130398 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e1ec9-f396-47dd-81f2-30faf6f40040 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:10.130398 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e1ec9-f396-47dd-81f2-30faf6f40040 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:10.131743 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 11/22] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:10.138322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d52f2-0826-4af0-b331-3536ce2e95ad None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:10.139331 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d52f2-0826-4af0-b331-3536ce2e95ad None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:10.139830 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d52f2-0826-4af0-b331-3536ce2e95ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:10.172215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ed6d52f2-0826-4af0-b331-3536ce2e95ad None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:10.174113 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ed6d52f2-0826-4af0-b331-3536ce2e95ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:10.198116 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acb85e-2ba1-410a-b2a5-a14f139bf99f None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:10.198435 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acb85e-2ba1-410a-b2a5-a14f139bf99f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:10.198665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acb85e-2ba1-410a-b2a5-a14f139bf99f None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:10.200718 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 12/23] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:11:10.207593 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dd9f2-eb0d-4624-9e47-b6cbabd1a8da None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:10.208448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dd9f2-eb0d-4624-9e47-b6cbabd1a8da None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:10.208681 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dd9f2-eb0d-4624-9e47-b6cbabd1a8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:10.239513 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a74dd9f2-eb0d-4624-9e47-b6cbabd1a8da None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:10.240740 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a74dd9f2-eb0d-4624-9e47-b6cbabd1a8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:10.301901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 12/24] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:10.307802 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6791a73-2f17-4d59-bc4a-b992960b3915 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:10.308097 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6791a73-2f17-4d59-bc4a-b992960b3915 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:10.308373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6791a73-2f17-4d59-bc4a-b992960b3915 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:10.314483 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 13/25] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:10 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:10.321012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e282c7-a613-4d14-bbd4-7058416afab3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:10.321286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e282c7-a613-4d14-bbd4-7058416afab3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:10.321558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e282c7-a613-4d14-bbd4-7058416afab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:10.352241 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-81e282c7-a613-4d14-bbd4-7058416afab3 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:10.353316 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 13/26] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:10.353754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-81e282c7-a613-4d14-bbd4-7058416afab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:10.457151 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:10.461876 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 14/27] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:11:10.649728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dd9f2-eb0d-4624-9e47-b6cbabd1a8da None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:10.663187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:10.708291 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3dfb-e64e-4501-a0de-56640f95dcc2 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:10.708291 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3dfb-e64e-4501-a0de-56640f95dcc2 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:10.708526 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3dfb-e64e-4501-a0de-56640f95dcc2 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:10.709904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 15/28] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:10.714706 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6ff4d-8e74-45aa-8f39-c8b6ea0a718e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:10.714991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6ff4d-8e74-45aa-8f39-c8b6ea0a718e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:10.715304 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6ff4d-8e74-45aa-8f39-c8b6ea0a718e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:10.755224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-61e6ff4d-8e74-45aa-8f39-c8b6ea0a718e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:10.756545 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-61e6ff4d-8e74-45aa-8f39-c8b6ea0a718e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:10.769099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dd9f2-eb0d-4624-9e47-b6cbabd1a8da None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:10.806428 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c1c99-389c-4f84-870b-2c0209626d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:10.807512 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c1c99-389c-4f84-870b-2c0209626d0c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:10.808062 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c1c99-389c-4f84-870b-2c0209626d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:10.808344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c1c99-389c-4f84-870b-2c0209626d0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:10.819968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c1c99-389c-4f84-870b-2c0209626d0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:10.821551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c1c99-389c-4f84-870b-2c0209626d0c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-a28c1c99-389c-4f84-870b-2c0209626d0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:11:10.837446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:10.838287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:11:10.839104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:11:10.839104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:11:10.839104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:10.839104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 14/29] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:10.839766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c1c99-389c-4f84-870b-2c0209626d0c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:10.879807 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 16/30] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:11:10.887814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6ff4d-8e74-45aa-8f39-c8b6ea0a718e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:10.907278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:10.951229 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c1c99-389c-4f84-870b-2c0209626d0c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:10.988055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1414ed-da23-45b7-aea3-14d13456a297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:10.988623 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1414ed-da23-45b7-aea3-14d13456a297 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:10.988903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1414ed-da23-45b7-aea3-14d13456a297 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:10.989958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1414ed-da23-45b7-aea3-14d13456a297 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:11.002385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1414ed-da23-45b7-aea3-14d13456a297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-3d1414ed-da23-45b7-aea3-14d13456a297 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:11.009761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:11.012052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:11.013921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:11.013921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:11.013921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:11.013921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:11.013921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:11.013921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:11.013921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:11.013921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:11.013921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:11.013921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:11.013921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:11:11.013921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:11.013921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 15/31] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:11:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:11.013921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1414ed-da23-45b7-aea3-14d13456a297 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:11.089262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6ff4d-8e74-45aa-8f39-c8b6ea0a718e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:11.129452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1414ed-da23-45b7-aea3-14d13456a297 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:11.152727 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:11.158221 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:11.165434 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35cd14-af8c-4b94-986c-20bedaf1db8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:11.166071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35cd14-af8c-4b94-986c-20bedaf1db8c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:11.166404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35cd14-af8c-4b94-986c-20bedaf1db8c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:11.166687 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35cd14-af8c-4b94-986c-20bedaf1db8c None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:11.167385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35cd14-af8c-4b94-986c-20bedaf1db8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:11.167690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35cd14-af8c-4b94-986c-20bedaf1db8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:11.169214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35cd14-af8c-4b94-986c-20bedaf1db8c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:11.173373 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f7f46-b9f5-4419-aac1-3f9be091fbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:11.174011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f7f46-b9f5-4419-aac1-3f9be091fbe2 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:11.174296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f7f46-b9f5-4419-aac1-3f9be091fbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:11.174575 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f7f46-b9f5-4419-aac1-3f9be091fbe2 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:11.175279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f7f46-b9f5-4419-aac1-3f9be091fbe2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:11.176668 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f7f46-b9f5-4419-aac1-3f9be091fbe2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:11.206459 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 17/32] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:11.207097 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35cd14-af8c-4b94-986c-20bedaf1db8c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:11.321929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 16/33] 10.222.0.18 () {60 vars in 1131 bytes} [Fri Oct 14 02:11:11 2022] POST /identity/v3/roles => generated 219 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:11.322733 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f7f46-b9f5-4419-aac1-3f9be091fbe2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:11.324402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35cd14-af8c-4b94-986c-20bedaf1db8c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:11.369242 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5171e-2b3b-499e-b256-1db8ad1d114d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:11.370037 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5171e-2b3b-499e-b256-1db8ad1d114d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:11.370329 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5171e-2b3b-499e-b256-1db8ad1d114d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:11.370608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5171e-2b3b-499e-b256-1db8ad1d114d None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:11.371332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5171e-2b3b-499e-b256-1db8ad1d114d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:11.371643 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5171e-2b3b-499e-b256-1db8ad1d114d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:11.373160 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5171e-2b3b-499e-b256-1db8ad1d114d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:11.385362 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 18/34] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:11.386232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5171e-2b3b-499e-b256-1db8ad1d114d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:11.467738 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:11.554789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5171e-2b3b-499e-b256-1db8ad1d114d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:11.565646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f7f46-b9f5-4419-aac1-3f9be091fbe2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:11.607513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad11a0-abda-4b28-be93-57b10ebbd98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:11.608313 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad11a0-abda-4b28-be93-57b10ebbd98e None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:11.608605 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad11a0-abda-4b28-be93-57b10ebbd98e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:11.609288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad11a0-abda-4b28-be93-57b10ebbd98e None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:11.609986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad11a0-abda-4b28-be93-57b10ebbd98e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:11.612039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad11a0-abda-4b28-be93-57b10ebbd98e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:11.615492 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945510b-3568-4574-8a46-0bdb585efd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:11.616139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945510b-3568-4574-8a46-0bdb585efd2b None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:11.616425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945510b-3568-4574-8a46-0bdb585efd2b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:11.616703 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945510b-3568-4574-8a46-0bdb585efd2b None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:11.617375 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945510b-3568-4574-8a46-0bdb585efd2b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:11.619022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945510b-3568-4574-8a46-0bdb585efd2b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:11.637094 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 19/35] 10.222.0.18 () {60 vars in 1131 bytes} [Fri Oct 14 02:11:11 2022] POST /identity/v3/roles => generated 225 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:11.638284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad11a0-abda-4b28-be93-57b10ebbd98e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:11.661636 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 17/36] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Oct 14 02:11:11 2022] POST /identity/v3/projects => generated 301 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:11.844230 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a80245-a9af-458d-b2d8-7ad52c961ace None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:11.844693 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a80245-a9af-458d-b2d8-7ad52c961ace None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:11.844978 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a80245-a9af-458d-b2d8-7ad52c961ace None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:11.846664 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 18/37] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:11.852894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a95bf3-4c91-468c-885f-8a04c2e52437 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:11.853289 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a95bf3-4c91-468c-885f-8a04c2e52437 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:11.853571 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a95bf3-4c91-468c-885f-8a04c2e52437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:11.873927 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-77a95bf3-4c91-468c-885f-8a04c2e52437 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:11.875752 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-77a95bf3-4c91-468c-885f-8a04c2e52437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:11.897027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad11a0-abda-4b28-be93-57b10ebbd98e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:11.907204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:11.926406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:11.957595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dcc08-1cef-4033-8137-308f9e60f65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:11.958377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dcc08-1cef-4033-8137-308f9e60f65d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:11.958673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dcc08-1cef-4033-8137-308f9e60f65d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:11.958948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dcc08-1cef-4033-8137-308f9e60f65d None admin] PATH_INFO: `/v3/users/6f436f18778d45bb9d581131675fcbb4` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:11.960698 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8dcc08-1cef-4033-8137-308f9e60f65d None admin] RBAC: Authorizing `identity:get_user(user_id=6f436f18778d45bb9d581131675fcbb4)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:11.962161 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8dcc08-1cef-4033-8137-308f9e60f65d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:11.964000 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 20/38] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:11 2022] GET /identity/v3/users/6f436f18778d45bb9d581131675fcbb4 => generated 247 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:11.969635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dcc08-1cef-4033-8137-308f9e60f65d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:11.971765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 19/39] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:11:12.081145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dcc08-1cef-4033-8137-308f9e60f65d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:12.121428 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9cf72-2be4-438e-8df1-96cea81556d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:12.122108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9cf72-2be4-438e-8df1-96cea81556d4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:12.122392 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9cf72-2be4-438e-8df1-96cea81556d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:12.122666 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9cf72-2be4-438e-8df1-96cea81556d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:12.123819 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9cf72-2be4-438e-8df1-96cea81556d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:12.125613 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9cf72-2be4-438e-8df1-96cea81556d4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:12.127199 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 21/40] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:12.133876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a95bf3-4c91-468c-885f-8a04c2e52437 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:12.213613 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7e292-961a-41f3-98d8-04eb636727fa None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:12.214115 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7e292-961a-41f3-98d8-04eb636727fa None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:12.214186 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7e292-961a-41f3-98d8-04eb636727fa None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:12.215570 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 22/41] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:11:12.220468 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7c650-578b-4248-807e-f34323369734 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:12.220923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7c650-578b-4248-807e-f34323369734 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:12.221208 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7c650-578b-4248-807e-f34323369734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:12.240144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f4d7c650-578b-4248-807e-f34323369734 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:12.241355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f4d7c650-578b-4248-807e-f34323369734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:12.241755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a95bf3-4c91-468c-885f-8a04c2e52437 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:12.276600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9066bc-fbec-45b9-b55e-b483cac0dd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:12.277218 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9066bc-fbec-45b9-b55e-b483cac0dd77 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:12.277504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9066bc-fbec-45b9-b55e-b483cac0dd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:12.277779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9066bc-fbec-45b9-b55e-b483cac0dd77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:12.278510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9066bc-fbec-45b9-b55e-b483cac0dd77 None admin] RBAC: Adding query filter params (role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b, scope.domain.id=default, user.id=6f436f18778d45bb9d581131675fcbb4) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:12.278819 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9066bc-fbec-45b9-b55e-b483cac0dd77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:12.280301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9066bc-fbec-45b9-b55e-b483cac0dd77 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:12.286345 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:12.289134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 20/42] 10.222.0.18 () {56 vars in 1322 bytes} [Fri Oct 14 02:11:12 2022] GET /identity/v3/role_assignments?user.id=6f436f18778d45bb9d581131675fcbb4&scope.domain.id=default&role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:12.290083 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9066bc-fbec-45b9-b55e-b483cac0dd77 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:12.315328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 23/43] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:11:12.322843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:12.402771 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9066bc-fbec-45b9-b55e-b483cac0dd77 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:12.439305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a88feb-60e4-4ee3-9474-f65d0234cc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:12.439943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a88feb-60e4-4ee3-9474-f65d0234cc63 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:12.440228 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a88feb-60e4-4ee3-9474-f65d0234cc63 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:12.440503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a88feb-60e4-4ee3-9474-f65d0234cc63 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:12.441181 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a88feb-60e4-4ee3-9474-f65d0234cc63 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:12.443457 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a88feb-60e4-4ee3-9474-f65d0234cc63 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:12.459615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 21/44] 10.222.0.18 () {60 vars in 1131 bytes} [Fri Oct 14 02:11:12 2022] POST /identity/v3/roles => generated 223 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:12.587651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7c650-578b-4248-807e-f34323369734 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:12.689063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7c650-578b-4248-807e-f34323369734 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:12.722278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29323feb-ce38-43c5-95e3-3c19c8458b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:12.722912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29323feb-ce38-43c5-95e3-3c19c8458b79 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:12.723842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29323feb-ce38-43c5-95e3-3c19c8458b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:12.724052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29323feb-ce38-43c5-95e3-3c19c8458b79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:12.725174 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29323feb-ce38-43c5-95e3-3c19c8458b79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:12.726646 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29323feb-ce38-43c5-95e3-3c19c8458b79 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:12.728260 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 24/45] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:12.733936 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a88feb-60e4-4ee3-9474-f65d0234cc63 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:12.818964 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:12.831519 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a88feb-60e4-4ee3-9474-f65d0234cc63 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:12.863289 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc20256-985b-4113-aa98-88f7523394d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:12.864084 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc20256-985b-4113-aa98-88f7523394d4 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:12.864370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc20256-985b-4113-aa98-88f7523394d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:12.864641 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc20256-985b-4113-aa98-88f7523394d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:12.865292 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc20256-985b-4113-aa98-88f7523394d4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:12.866735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc20256-985b-4113-aa98-88f7523394d4 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:12.891871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 22/46] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Oct 14 02:11:12 2022] POST /identity/v3/projects => generated 312 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:15.002171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c6ad4-38e1-4ae1-afd5-b8f796b1ecd5 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:15.002669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c6ad4-38e1-4ae1-afd5-b8f796b1ecd5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:15.002750 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c6ad4-38e1-4ae1-afd5-b8f796b1ecd5 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:15.004125 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 25/47] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:15.010172 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5498ff6-a1f0-44e2-bf0d-7c33741b3aff None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:15.010558 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5498ff6-a1f0-44e2-bf0d-7c33741b3aff None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:15.010736 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5498ff6-a1f0-44e2-bf0d-7c33741b3aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:15.029560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b5498ff6-a1f0-44e2-bf0d-7c33741b3aff None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:15.030754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b5498ff6-a1f0-44e2-bf0d-7c33741b3aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:15.093998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 23/48] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:15.365350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c6ad4-38e1-4ae1-afd5-b8f796b1ecd5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:15.414688 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:15.468177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c6ad4-38e1-4ae1-afd5-b8f796b1ecd5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:15.500932 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be85b09-0ba1-4033-b04b-1c36c87ab904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:15.501556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be85b09-0ba1-4033-b04b-1c36c87ab904 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:15.501841 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be85b09-0ba1-4033-b04b-1c36c87ab904 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:15.502120 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be85b09-0ba1-4033-b04b-1c36c87ab904 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:15.503250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be85b09-0ba1-4033-b04b-1c36c87ab904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:15.505472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be85b09-0ba1-4033-b04b-1c36c87ab904 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:15.507024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 26/49] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:15.512594 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5498ff6-a1f0-44e2-bf0d-7c33741b3aff None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:15.578787 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:15.609123 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5498ff6-a1f0-44e2-bf0d-7c33741b3aff None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:15.642236 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e9f2e-243a-451f-8bb0-4c3b3c49833e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:15.642772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e9f2e-243a-451f-8bb0-4c3b3c49833e None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:15.643053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e9f2e-243a-451f-8bb0-4c3b3c49833e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:15.643357 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e9f2e-243a-451f-8bb0-4c3b3c49833e None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:15.643996 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e9f2e-243a-451f-8bb0-4c3b3c49833e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:15.645477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e9f2e-243a-451f-8bb0-4c3b3c49833e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:15.670533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 24/50] 10.222.0.18 () {60 vars in 1140 bytes} [Fri Oct 14 02:11:15 2022] POST /identity/v3/projects => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:17.798573 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa4ca3-cbf9-4c92-8ae0-59ffd17a492e None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:17.799058 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa4ca3-cbf9-4c92-8ae0-59ffd17a492e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:17.799190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa4ca3-cbf9-4c92-8ae0-59ffd17a492e None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:17.800493 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 27/51] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:17.806161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133ce26-d994-4bca-a5b5-c05b1916cf07 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:17.806572 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133ce26-d994-4bca-a5b5-c05b1916cf07 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:17.806712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133ce26-d994-4bca-a5b5-c05b1916cf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:17.825126 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b133ce26-d994-4bca-a5b5-c05b1916cf07 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:17.826298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b133ce26-d994-4bca-a5b5-c05b1916cf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:17.890355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 25/52] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:11:18.155672 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa4ca3-cbf9-4c92-8ae0-59ffd17a492e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:18.253928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa4ca3-cbf9-4c92-8ae0-59ffd17a492e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:18.267311 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:18.287624 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8107b9a-e9d8-4a11-ab5e-31287a4b3adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:18.288239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8107b9a-e9d8-4a11-ab5e-31287a4b3adf None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:18.288514 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8107b9a-e9d8-4a11-ab5e-31287a4b3adf None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:18.288787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8107b9a-e9d8-4a11-ab5e-31287a4b3adf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:18.289871 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8107b9a-e9d8-4a11-ab5e-31287a4b3adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:18.291325 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8107b9a-e9d8-4a11-ab5e-31287a4b3adf None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:18.292858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 28/53] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:18.298200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133ce26-d994-4bca-a5b5-c05b1916cf07 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:18.452605 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:18.500139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133ce26-d994-4bca-a5b5-c05b1916cf07 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:18.532629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97f2d5-aa2a-47df-b001-19e0715c254f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:18.533243 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97f2d5-aa2a-47df-b001-19e0715c254f None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:18.533528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97f2d5-aa2a-47df-b001-19e0715c254f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:18.533806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97f2d5-aa2a-47df-b001-19e0715c254f None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:18.534461 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97f2d5-aa2a-47df-b001-19e0715c254f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:18.536710 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97f2d5-aa2a-47df-b001-19e0715c254f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:18.562652 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 26/54] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:11:18 2022] POST /identity/v3/users => generated 275 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:21.822060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad6000-0a40-41a2-8dab-b7291cabd6e9 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:21.823011 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad6000-0a40-41a2-8dab-b7291cabd6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:21.823524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad6000-0a40-41a2-8dab-b7291cabd6e9 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:21.825092 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 29/55] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:21.830574 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d3974-9014-46f1-ae42-7bad45b8f542 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:21.831436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d3974-9014-46f1-ae42-7bad45b8f542 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:21.831926 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d3974-9014-46f1-ae42-7bad45b8f542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:21.854111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c55d3974-9014-46f1-ae42-7bad45b8f542 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:21.855353 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c55d3974-9014-46f1-ae42-7bad45b8f542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:21.945403 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 27/56] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:21.995861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410810a-0e08-4f83-8444-768508797758 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:21.996435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410810a-0e08-4f83-8444-768508797758 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:21.996902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410810a-0e08-4f83-8444-768508797758 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:21.998445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 30/57] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:22.014404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8b474-1d0e-4474-914f-76c05cc1ec08 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:22.014697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8b474-1d0e-4474-914f-76c05cc1ec08 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:22.014973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8b474-1d0e-4474-914f-76c05cc1ec08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:22.041956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-78c8b474-1d0e-4474-914f-76c05cc1ec08 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:22.043219 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-78c8b474-1d0e-4474-914f-76c05cc1ec08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:22.082733 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:22.137352 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 28/58] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:22.213972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e271095-2ed9-415d-9a12-70f29ed14f37 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:22.214337 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e271095-2ed9-415d-9a12-70f29ed14f37 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:22.214852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e271095-2ed9-415d-9a12-70f29ed14f37 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:22.216440 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 31/59] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:22.221966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2e7bb-48b4-4850-844f-1970977d534d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:22.222260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2e7bb-48b4-4850-844f-1970977d534d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:22.222544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2e7bb-48b4-4850-844f-1970977d534d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:22.248711 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e5e2e7bb-48b4-4850-844f-1970977d534d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:22.250182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e5e2e7bb-48b4-4850-844f-1970977d534d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:22.307395 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9780a0-828f-499e-ae4f-a490b655d505 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:22.308015 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9780a0-828f-499e-ae4f-a490b655d505 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:22.308488 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9780a0-828f-499e-ae4f-a490b655d505 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:22.310016 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 32/60] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:22.310904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9780a0-828f-499e-ae4f-a490b655d505 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:22.339765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 29/61] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:22.340866 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bda9f-d3ed-4005-9d96-4a7eff1aab39 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:22.341156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bda9f-d3ed-4005-9d96-4a7eff1aab39 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:22.341432 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bda9f-d3ed-4005-9d96-4a7eff1aab39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:22.380847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-689bda9f-d3ed-4005-9d96-4a7eff1aab39 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:22.382053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-689bda9f-d3ed-4005-9d96-4a7eff1aab39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:22.433462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9780a0-828f-499e-ae4f-a490b655d505 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:22.473521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc99c37-ed3e-484f-a9dd-9b64f5cba734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:22.474266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc99c37-ed3e-484f-a9dd-9b64f5cba734 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:22.474554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc99c37-ed3e-484f-a9dd-9b64f5cba734 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:22.474831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc99c37-ed3e-484f-a9dd-9b64f5cba734 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:22.486079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc99c37-ed3e-484f-a9dd-9b64f5cba734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-4fc99c37-ed3e-484f-a9dd-9b64f5cba734 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:22.491754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:22.492637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:22.493357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:22.493357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:22.493357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:22.493357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:22.493357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:22.493357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:22.493357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:22.493357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:22.493357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:22.493357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:22.493357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:22.493357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:22.498194 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 33/62] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:22.499547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc99c37-ed3e-484f-a9dd-9b64f5cba734 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:22.503480 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 30/63] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:22.506331 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bda9f-d3ed-4005-9d96-4a7eff1aab39 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:22.542189 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:22.553360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:22.615972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc99c37-ed3e-484f-a9dd-9b64f5cba734 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:22.652042 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10787d7a-5454-43f8-9b09-ba3782f44d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:22.652796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10787d7a-5454-43f8-9b09-ba3782f44d5f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:22.653177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10787d7a-5454-43f8-9b09-ba3782f44d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:22.653458 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10787d7a-5454-43f8-9b09-ba3782f44d5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:22.672859 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10787d7a-5454-43f8-9b09-ba3782f44d5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:22.677040 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bda9f-d3ed-4005-9d96-4a7eff1aab39 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-10787d7a-5454-43f8-9b09-ba3782f44d5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:22.679320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:22.680108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:22.680912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:22.680912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:22.680912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:22.680912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:22.680912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:22.680912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:22.680912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:22.680912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:22.680912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:22.680912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:22.680912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:11:22.680912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:22.681470 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 34/64] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:11:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:22.683107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10787d7a-5454-43f8-9b09-ba3782f44d5f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:22.730692 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae9c0d-d7fb-4c1e-b469-f05df375345d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:22.731414 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae9c0d-d7fb-4c1e-b469-f05df375345d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:22.731705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae9c0d-d7fb-4c1e-b469-f05df375345d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:22.731984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae9c0d-d7fb-4c1e-b469-f05df375345d None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:22.732728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae9c0d-d7fb-4c1e-b469-f05df375345d None admin] RBAC: Adding query filter params (name=member) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:22.733514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae9c0d-d7fb-4c1e-b469-f05df375345d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:22.734892 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae9c0d-d7fb-4c1e-b469-f05df375345d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:22.751129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 31/65] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:22.752515 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779a989-dce4-42cc-98e9-5c318b14caa3 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:22.752811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779a989-dce4-42cc-98e9-5c318b14caa3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:22.753092 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779a989-dce4-42cc-98e9-5c318b14caa3 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:22.755161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 32/66] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:22.756127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779a989-dce4-42cc-98e9-5c318b14caa3 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:22.807409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10787d7a-5454-43f8-9b09-ba3782f44d5f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:22.809840 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:22.857846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9d8bd-457d-4606-91fd-dc85a4f1d19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:22.858474 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9d8bd-457d-4606-91fd-dc85a4f1d19a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:22.858744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9d8bd-457d-4606-91fd-dc85a4f1d19a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:22.859018 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9d8bd-457d-4606-91fd-dc85a4f1d19a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:22.859548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:22.873107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9d8bd-457d-4606-91fd-dc85a4f1d19a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:22.874564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779a989-dce4-42cc-98e9-5c318b14caa3 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-ddf9d8bd-457d-4606-91fd-dc85a4f1d19a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:22.878378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:22.883187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:22.884013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:22.884013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:22.884013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:22.884013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:22.884013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:22.884013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:22.884013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:22.884013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:22.884013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:22.884013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:22.884013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:11:22.884013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:22.884013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 35/67] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:11:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:22.884013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9d8bd-457d-4606-91fd-dc85a4f1d19a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:22.913928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852694f-5687-4e5f-a3ba-f85e25bd4a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:22.914553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852694f-5687-4e5f-a3ba-f85e25bd4a4c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:22.914837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852694f-5687-4e5f-a3ba-f85e25bd4a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:22.915153 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852694f-5687-4e5f-a3ba-f85e25bd4a4c None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:22.915832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852694f-5687-4e5f-a3ba-f85e25bd4a4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:22.916140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852694f-5687-4e5f-a3ba-f85e25bd4a4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:22.918052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852694f-5687-4e5f-a3ba-f85e25bd4a4c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:22.928103 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 33/68] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:22.933447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be427582-c865-4fce-bec6-29ba67a0f062 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:22.933741 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be427582-c865-4fce-bec6-29ba67a0f062 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:22.934022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be427582-c865-4fce-bec6-29ba67a0f062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:22.954791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-be427582-c865-4fce-bec6-29ba67a0f062 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:22.956066 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-be427582-c865-4fce-bec6-29ba67a0f062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:22.998283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9d8bd-457d-4606-91fd-dc85a4f1d19a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:23.033583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 34/69] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:23.035124 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be427582-c865-4fce-bec6-29ba67a0f062 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:23.055998 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdc7b1-155b-4d63-b7bf-983be599644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:23.056794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdc7b1-155b-4d63-b7bf-983be599644c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:23.057076 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdc7b1-155b-4d63-b7bf-983be599644c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:23.057803 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdc7b1-155b-4d63-b7bf-983be599644c None admin] PATH_INFO: `/v3/users/e989ab80fa194e5ca4a4e53f88ae8ecb` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:23.074034 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdc7b1-155b-4d63-b7bf-983be599644c None admin] RBAC: Authorizing `identity:get_user(user_id=e989ab80fa194e5ca4a4e53f88ae8ecb)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:23.075623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdc7b1-155b-4d63-b7bf-983be599644c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:23.081694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 36/70] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:22 2022] GET /identity/v3/users/e989ab80fa194e5ca4a4e53f88ae8ecb => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:23.081694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdc7b1-155b-4d63-b7bf-983be599644c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:23.107961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:23.135916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:23.157567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be427582-c865-4fce-bec6-29ba67a0f062 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:23.202784 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdc7b1-155b-4d63-b7bf-983be599644c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:23.205186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56e03d-f2fe-40f7-a64e-403bac73030b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:23.205855 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56e03d-f2fe-40f7-a64e-403bac73030b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:23.206143 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56e03d-f2fe-40f7-a64e-403bac73030b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:23.206422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56e03d-f2fe-40f7-a64e-403bac73030b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:23.218678 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56e03d-f2fe-40f7-a64e-403bac73030b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-ff56e03d-f2fe-40f7-a64e-403bac73030b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:23.225536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:23.226379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:23.231801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:23.231801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:23.231801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:23.231801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:23.231801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:23.231801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:23.231801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:23.231801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:23.231801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:23.231801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:23.231801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:23.231801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:23.231801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 35/71] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:23.231801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56e03d-f2fe-40f7-a64e-403bac73030b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:23.252387 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb94466-7c50-4ee6-8c44-d274e7304299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:23.253339 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb94466-7c50-4ee6-8c44-d274e7304299 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:23.253814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb94466-7c50-4ee6-8c44-d274e7304299 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:23.254781 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb94466-7c50-4ee6-8c44-d274e7304299 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:23.255578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb94466-7c50-4ee6-8c44-d274e7304299 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:23.255661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb94466-7c50-4ee6-8c44-d274e7304299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:23.257554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb94466-7c50-4ee6-8c44-d274e7304299 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:23.267363 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 37/72] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:11:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:23.268195 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb94466-7c50-4ee6-8c44-d274e7304299 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:23.361135 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:23.372417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56e03d-f2fe-40f7-a64e-403bac73030b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:23.383089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb94466-7c50-4ee6-8c44-d274e7304299 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:23.414638 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a72b3-de7d-4091-979d-fd597f885995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:23.415300 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a72b3-de7d-4091-979d-fd597f885995 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:23.415583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a72b3-de7d-4091-979d-fd597f885995 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:23.415861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a72b3-de7d-4091-979d-fd597f885995 None admin] PATH_INFO: `/v3/users/6f436f18778d45bb9d581131675fcbb4` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:23.417636 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a72b3-de7d-4091-979d-fd597f885995 None admin] RBAC: Authorizing `identity:get_user(user_id=6f436f18778d45bb9d581131675fcbb4)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:23.419643 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82a8dd-be33-4b9b-8aaa-e34eb2a464ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:23.420037 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a72b3-de7d-4091-979d-fd597f885995 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:23.420339 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82a8dd-be33-4b9b-8aaa-e34eb2a464ed None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:23.420628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82a8dd-be33-4b9b-8aaa-e34eb2a464ed None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:23.420902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82a8dd-be33-4b9b-8aaa-e34eb2a464ed None admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:23.422031 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82a8dd-be33-4b9b-8aaa-e34eb2a464ed None admin] RBAC: Authorizing `identity:get_project(project_id=dabc91be3a7240e8bab6128f948854f0)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:23.422747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 36/73] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:23 2022] GET /identity/v3/users/6f436f18778d45bb9d581131675fcbb4 => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:23.423507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82a8dd-be33-4b9b-8aaa-e34eb2a464ed None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:23.424093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a72b3-de7d-4091-979d-fd597f885995 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:23.430842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 38/74] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:23 2022] GET /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:23.430842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82a8dd-be33-4b9b-8aaa-e34eb2a464ed None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:23.439209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:23.537012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a72b3-de7d-4091-979d-fd597f885995 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:23.538098 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82a8dd-be33-4b9b-8aaa-e34eb2a464ed None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:23.572072 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b7f44-20ba-42d7-9939-7686d3843e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:23.572280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087c1da-de73-430f-aa66-59fbf579a95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:23.572705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b7f44-20ba-42d7-9939-7686d3843e4d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:23.572898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087c1da-de73-430f-aa66-59fbf579a95c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:23.572984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b7f44-20ba-42d7-9939-7686d3843e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:23.573184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087c1da-de73-430f-aa66-59fbf579a95c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:23.573274 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b7f44-20ba-42d7-9939-7686d3843e4d None admin] PATH_INFO: `/v3/users/e989ab80fa194e5ca4a4e53f88ae8ecb` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:23.573466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087c1da-de73-430f-aa66-59fbf579a95c None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:23.574138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087c1da-de73-430f-aa66-59fbf579a95c None admin] RBAC: Adding query filter params (name=member) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:23.574363 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b7f44-20ba-42d7-9939-7686d3843e4d None admin] RBAC: Authorizing `identity:get_user(user_id=e989ab80fa194e5ca4a4e53f88ae8ecb)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:23.574465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087c1da-de73-430f-aa66-59fbf579a95c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:23.575842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b7f44-20ba-42d7-9939-7686d3843e4d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:23.575936 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087c1da-de73-430f-aa66-59fbf579a95c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:23.577433 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 39/75] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:23 2022] GET /identity/v3/users/e989ab80fa194e5ca4a4e53f88ae8ecb => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:23.578198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b7f44-20ba-42d7-9939-7686d3843e4d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:23.593183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 37/76] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:23.593183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087c1da-de73-430f-aa66-59fbf579a95c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:23.601847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:23.658158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:23.682169 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b7f44-20ba-42d7-9939-7686d3843e4d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:23.687941 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087c1da-de73-430f-aa66-59fbf579a95c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:23.715475 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805a735-a80e-4c43-8245-59c0607e3cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:23.716102 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805a735-a80e-4c43-8245-59c0607e3cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:23.716384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805a735-a80e-4c43-8245-59c0607e3cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:23.716658 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805a735-a80e-4c43-8245-59c0607e3cb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:23.717775 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805a735-a80e-4c43-8245-59c0607e3cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:23.719258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805a735-a80e-4c43-8245-59c0607e3cb7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:23.720621 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4136c5-9825-40a3-ab95-0235c381b3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:23.721147 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 40/77] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:23.721250 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4136c5-9825-40a3-ab95-0235c381b3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:23.721533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4136c5-9825-40a3-ab95-0235c381b3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:23.721724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805a735-a80e-4c43-8245-59c0607e3cb7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:23.721950 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4136c5-9825-40a3-ab95-0235c381b3e5 None admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:23.723922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4136c5-9825-40a3-ab95-0235c381b3e5 None admin] RBAC: Authorizing `identity:get_project(project_id=dabc91be3a7240e8bab6128f948854f0)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:23.725359 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4136c5-9825-40a3-ab95-0235c381b3e5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:23.727623 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 38/78] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:23 2022] GET /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:23.727804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4136c5-9825-40a3-ab95-0235c381b3e5 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:23.812981 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:23.827873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805a735-a80e-4c43-8245-59c0607e3cb7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:23.830865 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4136c5-9825-40a3-ab95-0235c381b3e5 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:23.859561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f0f0c-33f9-4651-822f-40d359d9b7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:23.860217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f0f0c-33f9-4651-822f-40d359d9b7d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:23.860496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f0f0c-33f9-4651-822f-40d359d9b7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:23.860770 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f0f0c-33f9-4651-822f-40d359d9b7d4 None admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0/users/e989ab80fa194e5ca4a4e53f88ae8ecb/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:23.862639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f0f0c-33f9-4651-822f-40d359d9b7d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=dabc91be3a7240e8bab6128f948854f0, user_id=e989ab80fa194e5ca4a4e53f88ae8ecb, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:23.862926 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926808e-e8d5-4c93-949d-412246d51b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:23.864109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926808e-e8d5-4c93-949d-412246d51b8c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:23.864395 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926808e-e8d5-4c93-949d-412246d51b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:23.864672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926808e-e8d5-4c93-949d-412246d51b8c None admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:23.864754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f0f0c-33f9-4651-822f-40d359d9b7d4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:23.865808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926808e-e8d5-4c93-949d-412246d51b8c None admin] RBAC: Authorizing `identity:get_project(project_id=dabc91be3a7240e8bab6128f948854f0)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:23.867283 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926808e-e8d5-4c93-949d-412246d51b8c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:23.868958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 39/79] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:23 2022] GET /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:23.869833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926808e-e8d5-4c93-949d-412246d51b8c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:23.878278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 41/80] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:11:23 2022] PUT /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0/users/e989ab80fa194e5ca4a4e53f88ae8ecb/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:11:23.879458 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f0f0c-33f9-4651-822f-40d359d9b7d4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:23.888618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:23.992986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926808e-e8d5-4c93-949d-412246d51b8c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:23.996099 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f0f0c-33f9-4651-822f-40d359d9b7d4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:24.033807 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92174745-b3ae-450b-be81-a98de4793746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:24.034020 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1e6ce-fd02-407e-a4d3-7fde213d6198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:24.034471 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174745-b3ae-450b-be81-a98de4793746 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:24.034590 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1e6ce-fd02-407e-a4d3-7fde213d6198 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:24.034757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174745-b3ae-450b-be81-a98de4793746 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:24.034847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1e6ce-fd02-407e-a4d3-7fde213d6198 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:24.035562 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174745-b3ae-450b-be81-a98de4793746 None admin] PATH_INFO: `/v3/users/e989ab80fa194e5ca4a4e53f88ae8ecb` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:24.035719 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1e6ce-fd02-407e-a4d3-7fde213d6198 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:24.036427 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1e6ce-fd02-407e-a4d3-7fde213d6198 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:24.036930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92174745-b3ae-450b-be81-a98de4793746 None admin] RBAC: Authorizing `identity:get_user(user_id=e989ab80fa194e5ca4a4e53f88ae8ecb)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:24.037916 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1e6ce-fd02-407e-a4d3-7fde213d6198 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:24.038405 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92174745-b3ae-450b-be81-a98de4793746 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:24.044865 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 40/81] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:23 2022] GET /identity/v3/users/e989ab80fa194e5ca4a4e53f88ae8ecb => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:24.044865 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92174745-b3ae-450b-be81-a98de4793746 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:24.077017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 42/82] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:11:23 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:24.078438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1e6ce-fd02-407e-a4d3-7fde213d6198 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:24.081475 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:24.149697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:24.180287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92174745-b3ae-450b-be81-a98de4793746 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:24.224203 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1e6ce-fd02-407e-a4d3-7fde213d6198 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:24.228460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3deb55f-49c4-4dce-b922-58dfcc738f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:24.229453 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deb55f-49c4-4dce-b922-58dfcc738f0b None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:24.229926 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deb55f-49c4-4dce-b922-58dfcc738f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:24.230401 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deb55f-49c4-4dce-b922-58dfcc738f0b None admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0/users/6f436f18778d45bb9d581131675fcbb4/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:24.234133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3deb55f-49c4-4dce-b922-58dfcc738f0b None admin] RBAC: Authorizing `identity:create_grant(project_id=dabc91be3a7240e8bab6128f948854f0, user_id=6f436f18778d45bb9d581131675fcbb4, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:24.236411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3deb55f-49c4-4dce-b922-58dfcc738f0b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:24.262962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 41/83] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:11:24 2022] PUT /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0/users/6f436f18778d45bb9d581131675fcbb4/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:11:24.263846 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3deb55f-49c4-4dce-b922-58dfcc738f0b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:24.268241 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae73c4f-827b-48ef-a2ec-68d311109dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:24.268942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae73c4f-827b-48ef-a2ec-68d311109dc8 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:24.269226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae73c4f-827b-48ef-a2ec-68d311109dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:24.270063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae73c4f-827b-48ef-a2ec-68d311109dc8 None admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0/users/e989ab80fa194e5ca4a4e53f88ae8ecb/roles/a3287688078248ecb19a6d9c85839631` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:24.276081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae73c4f-827b-48ef-a2ec-68d311109dc8 None admin] RBAC: Authorizing `identity:create_grant(project_id=dabc91be3a7240e8bab6128f948854f0, user_id=e989ab80fa194e5ca4a4e53f88ae8ecb, role_id=a3287688078248ecb19a6d9c85839631)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:24.278937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae73c4f-827b-48ef-a2ec-68d311109dc8 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:24.297804 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 43/84] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:11:24 2022] PUT /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0/users/e989ab80fa194e5ca4a4e53f88ae8ecb/roles/a3287688078248ecb19a6d9c85839631 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:11:24.405625 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3deb55f-49c4-4dce-b922-58dfcc738f0b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:24.449921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ce7e1-695d-43ac-9df3-a4ad2e901710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:24.450598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ce7e1-695d-43ac-9df3-a4ad2e901710 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:24.450886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ce7e1-695d-43ac-9df3-a4ad2e901710 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:24.451220 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ce7e1-695d-43ac-9df3-a4ad2e901710 None admin] PATH_INFO: `/v3/projects/d6ba3ca61bd54d71a217b379e39a90e5` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:24.452491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ce7e1-695d-43ac-9df3-a4ad2e901710 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ba3ca61bd54d71a217b379e39a90e5)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:24.454116 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ce7e1-695d-43ac-9df3-a4ad2e901710 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:24.456429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 42/85] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:24 2022] GET /identity/v3/projects/d6ba3ca61bd54d71a217b379e39a90e5 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:24.462450 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae73c4f-827b-48ef-a2ec-68d311109dc8 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:24.534019 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:24.576794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae73c4f-827b-48ef-a2ec-68d311109dc8 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:24.612537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bbe07-9259-4af8-b3aa-3172b4b558eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:24.614198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bbe07-9259-4af8-b3aa-3172b4b558eb None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:24.614707 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bbe07-9259-4af8-b3aa-3172b4b558eb None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:24.615202 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bbe07-9259-4af8-b3aa-3172b4b558eb None admin] PATH_INFO: `/v3/projects/d6ba3ca61bd54d71a217b379e39a90e5/users/e989ab80fa194e5ca4a4e53f88ae8ecb/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:24.617351 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93bbe07-9259-4af8-b3aa-3172b4b558eb None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ba3ca61bd54d71a217b379e39a90e5, user_id=e989ab80fa194e5ca4a4e53f88ae8ecb, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:24.618996 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93bbe07-9259-4af8-b3aa-3172b4b558eb None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:24.637992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 44/86] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:11:24 2022] PUT /identity/v3/projects/d6ba3ca61bd54d71a217b379e39a90e5/users/e989ab80fa194e5ca4a4e53f88ae8ecb/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:11:26.935263 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd373c4-4827-4b28-b1c1-dd9c52b4ad31 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:26.936288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd373c4-4827-4b28-b1c1-dd9c52b4ad31 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:26.936781 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd373c4-4827-4b28-b1c1-dd9c52b4ad31 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:26.938645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 43/87] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:26.944386 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce44082-f714-4f27-8448-54a087aa13a1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:26.945185 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce44082-f714-4f27-8448-54a087aa13a1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:26.945669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce44082-f714-4f27-8448-54a087aa13a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:26.967407 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-cce44082-f714-4f27-8448-54a087aa13a1 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:26.968635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cce44082-f714-4f27-8448-54a087aa13a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:27.053573 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 45/88] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:27.172158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cb7d2-c02e-491e-a1a9-f92b8bf64b56 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:27.172783 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cb7d2-c02e-491e-a1a9-f92b8bf64b56 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:27.173261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cb7d2-c02e-491e-a1a9-f92b8bf64b56 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:27.174833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 44/89] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:27.180466 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ade0d-a9cc-4d72-9988-37e8fe5bbc39 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:27.180757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ade0d-a9cc-4d72-9988-37e8fe5bbc39 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:27.181036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ade0d-a9cc-4d72-9988-37e8fe5bbc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:27.202714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ea3ade0d-a9cc-4d72-9988-37e8fe5bbc39 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:27.203930 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ea3ade0d-a9cc-4d72-9988-37e8fe5bbc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:27.261334 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:27.284519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 46/90] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:27.588204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee0513-1eb3-4670-9d4c-d899bddf4693 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:27.588888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee0513-1eb3-4670-9d4c-d899bddf4693 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:27.589378 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee0513-1eb3-4670-9d4c-d899bddf4693 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:27.590945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 45/91] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:27.604526 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243a1bc-bf6a-43f4-9fcb-063ddeb605de None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:27.604802 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243a1bc-bf6a-43f4-9fcb-063ddeb605de None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:27.605087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243a1bc-bf6a-43f4-9fcb-063ddeb605de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:27.612377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee0513-1eb3-4670-9d4c-d899bddf4693 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:27.618041 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:27.634494 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3243a1bc-bf6a-43f4-9fcb-063ddeb605de None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:27.636344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3243a1bc-bf6a-43f4-9fcb-063ddeb605de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:27.782405 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee0513-1eb3-4670-9d4c-d899bddf4693 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:27.784704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 47/92] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:27.788146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243a1bc-bf6a-43f4-9fcb-063ddeb605de None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:27.838694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3868d-decc-4565-9aae-4fef81246604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:27.839287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3868d-decc-4565-9aae-4fef81246604 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:27.839624 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3868d-decc-4565-9aae-4fef81246604 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:27.839901 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3868d-decc-4565-9aae-4fef81246604 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:27.851789 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3868d-decc-4565-9aae-4fef81246604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-eec3868d-decc-4565-9aae-4fef81246604 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:27.857285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:27.858107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:27.859255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:27.859255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:27.859255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:27.859255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:27.859255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:27.859255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:27.859255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:27.859255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:27.859255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:27.859255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:27.859255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:27.859255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:27.859255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 46/93] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:11:27.860100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3a7c1-d91f-4e32-829c-b190b18033ce None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:27.860100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3a7c1-d91f-4e32-829c-b190b18033ce None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:27.860367 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3a7c1-d91f-4e32-829c-b190b18033ce None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:27.861617 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 47/94] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:27.862680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25a5d4-c9ce-46be-bbbc-8f134246c85a None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:27.862936 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25a5d4-c9ce-46be-bbbc-8f134246c85a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:27.863806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25a5d4-c9ce-46be-bbbc-8f134246c85a None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:27.864528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 48/95] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:27.869249 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25a5d4-c9ce-46be-bbbc-8f134246c85a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:27.924494 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243a1bc-bf6a-43f4-9fcb-063ddeb605de None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:27.942555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:27.944652 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:27.972779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971e4fb-b6fb-470c-9fad-f35c19fc8088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:27.974648 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971e4fb-b6fb-470c-9fad-f35c19fc8088 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:27.975404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971e4fb-b6fb-470c-9fad-f35c19fc8088 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:27.975904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971e4fb-b6fb-470c-9fad-f35c19fc8088 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:27.987279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25a5d4-c9ce-46be-bbbc-8f134246c85a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:27.991283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971e4fb-b6fb-470c-9fad-f35c19fc8088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-9971e4fb-b6fb-470c-9fad-f35c19fc8088 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:27.998513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:27.999628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:28.000741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:28.000741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:28.000741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:28.000741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:28.000741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:28.000741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:28.000741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:28.000741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:28.000741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:28.000741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:28.000741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:11:28.000741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:28.000741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 48/96] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:11:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:28.002847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50751264-8bc9-40e2-b617-00ea10b12066 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:28.003160 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50751264-8bc9-40e2-b617-00ea10b12066 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:28.003440 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50751264-8bc9-40e2-b617-00ea10b12066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:28.021896 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-50751264-8bc9-40e2-b617-00ea10b12066 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:28.023194 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-50751264-8bc9-40e2-b617-00ea10b12066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:28.031257 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e25a3-380c-4b2f-90c4-99dcec86a13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:28.031881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e25a3-380c-4b2f-90c4-99dcec86a13a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:28.032170 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e25a3-380c-4b2f-90c4-99dcec86a13a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:28.032452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e25a3-380c-4b2f-90c4-99dcec86a13a None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:28.033133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e25a3-380c-4b2f-90c4-99dcec86a13a None admin] RBAC: Adding query filter params (name=member) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:28.033443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e25a3-380c-4b2f-90c4-99dcec86a13a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:28.034894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e25a3-380c-4b2f-90c4-99dcec86a13a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:28.043449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 49/97] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:28.044628 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338243b1-6499-4298-9e62-d13a984b77ee None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:28.044919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338243b1-6499-4298-9e62-d13a984b77ee None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:28.045199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338243b1-6499-4298-9e62-d13a984b77ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:28.064943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-338243b1-6499-4298-9e62-d13a984b77ee None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:28.067425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-338243b1-6499-4298-9e62-d13a984b77ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:28.097814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 49/98] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:28.098384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50751264-8bc9-40e2-b617-00ea10b12066 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:28.139753 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 50/99] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:28.149665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc36123-11a3-4837-bf77-50cb2fcff93c None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:28.149665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc36123-11a3-4837-bf77-50cb2fcff93c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:28.149665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc36123-11a3-4837-bf77-50cb2fcff93c None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:28.149665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 51/100] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:28.149665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc36123-11a3-4837-bf77-50cb2fcff93c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:28.222005 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:28.222182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50751264-8bc9-40e2-b617-00ea10b12066 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:28.224484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:28.272644 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc36123-11a3-4837-bf77-50cb2fcff93c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:28.272930 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe157387-8cf9-4894-81dc-0362ba097aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:28.273537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe157387-8cf9-4894-81dc-0362ba097aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:28.273831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe157387-8cf9-4894-81dc-0362ba097aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:28.274372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe157387-8cf9-4894-81dc-0362ba097aa5 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:28.275928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe157387-8cf9-4894-81dc-0362ba097aa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:28.276246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe157387-8cf9-4894-81dc-0362ba097aa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:28.277667 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe157387-8cf9-4894-81dc-0362ba097aa5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:28.289376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 50/101] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:28.290380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe157387-8cf9-4894-81dc-0362ba097aa5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:28.311867 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a12e9-b7e5-41e8-b18f-250a5b214884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:28.312506 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a12e9-b7e5-41e8-b18f-250a5b214884 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:28.312786 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a12e9-b7e5-41e8-b18f-250a5b214884 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:28.313058 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a12e9-b7e5-41e8-b18f-250a5b214884 None admin] PATH_INFO: `/v3/users/e989ab80fa194e5ca4a4e53f88ae8ecb` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:28.314237 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a12e9-b7e5-41e8-b18f-250a5b214884 None admin] RBAC: Authorizing `identity:get_user(user_id=e989ab80fa194e5ca4a4e53f88ae8ecb)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:28.316386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a12e9-b7e5-41e8-b18f-250a5b214884 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:28.318044 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 52/102] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:28 2022] GET /identity/v3/users/e989ab80fa194e5ca4a4e53f88ae8ecb => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:28.319227 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a4ddd-7b45-43ff-b009-ed1c91ae134a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:28.319507 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a4ddd-7b45-43ff-b009-ed1c91ae134a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:28.319778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a4ddd-7b45-43ff-b009-ed1c91ae134a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:28.341294 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-020a4ddd-7b45-43ff-b009-ed1c91ae134a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:28.343348 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-020a4ddd-7b45-43ff-b009-ed1c91ae134a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:28.417674 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe157387-8cf9-4894-81dc-0362ba097aa5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:28.420425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 53/103] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:28.420816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a4ddd-7b45-43ff-b009-ed1c91ae134a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:28.460447 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2dbba3-60d9-4c14-befc-8bbe7e0b5d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:28.461100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dbba3-60d9-4c14-befc-8bbe7e0b5d45 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:28.461385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dbba3-60d9-4c14-befc-8bbe7e0b5d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:28.461663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dbba3-60d9-4c14-befc-8bbe7e0b5d45 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:28.475885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2dbba3-60d9-4c14-befc-8bbe7e0b5d45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:28.479029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-2c2dbba3-60d9-4c14-befc-8bbe7e0b5d45 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:28.481244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:28.482397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:28.483476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:28.483476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:28.483476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:28.483476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:28.483476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:28.483476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:28.483476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:28.483476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:28.483476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:28.483476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:28.483476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:11:28.483476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:28.483476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 51/104] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:11:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:28.484187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2dbba3-60d9-4c14-befc-8bbe7e0b5d45 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:28.509422 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:28.533918 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a4ddd-7b45-43ff-b009-ed1c91ae134a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:28.572630 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c2a4e-7965-4700-b25d-47f5174401f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:28.573572 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c2a4e-7965-4700-b25d-47f5174401f6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:28.573858 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c2a4e-7965-4700-b25d-47f5174401f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:28.574138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c2a4e-7965-4700-b25d-47f5174401f6 None admin] PATH_INFO: `/v3/users/6f436f18778d45bb9d581131675fcbb4` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:28.575308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c2a4e-7965-4700-b25d-47f5174401f6 None admin] RBAC: Authorizing `identity:get_user(user_id=6f436f18778d45bb9d581131675fcbb4)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:28.577203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c2a4e-7965-4700-b25d-47f5174401f6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:28.578796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 54/105] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:28 2022] GET /identity/v3/users/6f436f18778d45bb9d581131675fcbb4 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:28.578796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c2a4e-7965-4700-b25d-47f5174401f6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:28.594611 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2dbba3-60d9-4c14-befc-8bbe7e0b5d45 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:28.632064 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff891e-303f-4b58-931b-b7814020b41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:28.632674 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff891e-303f-4b58-931b-b7814020b41c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:28.633148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff891e-303f-4b58-931b-b7814020b41c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:28.633250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff891e-303f-4b58-931b-b7814020b41c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:28.686007 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c2a4e-7965-4700-b25d-47f5174401f6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:28.703190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff891e-303f-4b58-931b-b7814020b41c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:28.704671 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff891e-303f-4b58-931b-b7814020b41c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:28.709694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:28.721196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c2b62-885d-419f-af50-61f3a554f121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:28.721839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c2b62-885d-419f-af50-61f3a554f121 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:28.722120 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c2b62-885d-419f-af50-61f3a554f121 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:28.722942 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c2b62-885d-419f-af50-61f3a554f121 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:28.729397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:28.733706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c2b62-885d-419f-af50-61f3a554f121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-5e0c2b62-885d-419f-af50-61f3a554f121 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:28.739090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:28.739933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:28.740663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:28.740663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:28.740663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:28.740663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:28.740663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:28.740663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:28.740663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:28.740663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:28.740663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:28.740663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:28.740663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:11:28.740663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:28.740663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 55/106] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:28.741269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c2b62-885d-419f-af50-61f3a554f121 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:28.806409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 52/107] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:11:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:11:28.807386 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff891e-303f-4b58-931b-b7814020b41c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:28.841733 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c2b62-885d-419f-af50-61f3a554f121 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:28.874524 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fc1a4-d810-4783-8109-8543af92d1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:28.875194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fc1a4-d810-4783-8109-8543af92d1fe None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:28.875476 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fc1a4-d810-4783-8109-8543af92d1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:28.875751 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fc1a4-d810-4783-8109-8543af92d1fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:28.876865 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1fc1a4-d810-4783-8109-8543af92d1fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:28.878308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1fc1a4-d810-4783-8109-8543af92d1fe None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:28.879939 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 56/108] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:28.880787 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fc1a4-d810-4783-8109-8543af92d1fe None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:28.907967 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff891e-303f-4b58-931b-b7814020b41c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:28.928987 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:28.931201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:28.943088 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468e4da-9942-494f-867b-fb6b274025f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:28.943650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468e4da-9942-494f-867b-fb6b274025f2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:28.943934 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468e4da-9942-494f-867b-fb6b274025f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:28.944213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468e4da-9942-494f-867b-fb6b274025f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:28.944894 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468e4da-9942-494f-867b-fb6b274025f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:28.945455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468e4da-9942-494f-867b-fb6b274025f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:28.946957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468e4da-9942-494f-867b-fb6b274025f2 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:28.954407 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 53/109] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:11:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:28.956033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468e4da-9942-494f-867b-fb6b274025f2 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:28.979924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fc1a4-d810-4783-8109-8543af92d1fe None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:29.014373 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924a0b5-d9c1-4042-8e74-4d3aa335e449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:29.014985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924a0b5-d9c1-4042-8e74-4d3aa335e449 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:29.015293 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924a0b5-d9c1-4042-8e74-4d3aa335e449 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:29.015564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924a0b5-d9c1-4042-8e74-4d3aa335e449 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:29.054228 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468e4da-9942-494f-867b-fb6b274025f2 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:29.081258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924a0b5-d9c1-4042-8e74-4d3aa335e449 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:29.083447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924a0b5-d9c1-4042-8e74-4d3aa335e449 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:29.087221 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad290e61-ea66-48b2-a4f0-50a263fa61cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:29.087844 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad290e61-ea66-48b2-a4f0-50a263fa61cf None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:29.088121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad290e61-ea66-48b2-a4f0-50a263fa61cf None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:29.088391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad290e61-ea66-48b2-a4f0-50a263fa61cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:29.098339 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad290e61-ea66-48b2-a4f0-50a263fa61cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-ad290e61-ea66-48b2-a4f0-50a263fa61cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:29.104307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:29.105127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:29.106024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:29.106024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:29.106024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:29.106024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:29.106024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:29.106024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:29.106024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:29.106024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:29.106024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:29.106024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:29.106024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:29.106024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:29.106024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 54/110] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:29.108030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad290e61-ea66-48b2-a4f0-50a263fa61cf None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:29.131635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:29.153122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:29.181929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 57/111] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:11:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:11:29.182777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924a0b5-d9c1-4042-8e74-4d3aa335e449 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:29.208853 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad290e61-ea66-48b2-a4f0-50a263fa61cf None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:29.241204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8b1bc-c9c0-49c4-b83a-93d998ca48c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:29.241814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8b1bc-c9c0-49c4-b83a-93d998ca48c8 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:29.242098 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8b1bc-c9c0-49c4-b83a-93d998ca48c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:29.242372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8b1bc-c9c0-49c4-b83a-93d998ca48c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:29.243157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8b1bc-c9c0-49c4-b83a-93d998ca48c8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:29.243469 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8b1bc-c9c0-49c4-b83a-93d998ca48c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:29.244867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8b1bc-c9c0-49c4-b83a-93d998ca48c8 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:29.253273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 55/112] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:29.254737 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8b1bc-c9c0-49c4-b83a-93d998ca48c8 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:29.283799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924a0b5-d9c1-4042-8e74-4d3aa335e449 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:29.317377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644c717-fc95-44fb-b0f7-98a64d7b6e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:29.318021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644c717-fc95-44fb-b0f7-98a64d7b6e18 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:29.318303 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644c717-fc95-44fb-b0f7-98a64d7b6e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:29.318584 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644c717-fc95-44fb-b0f7-98a64d7b6e18 None admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:29.319729 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644c717-fc95-44fb-b0f7-98a64d7b6e18 None admin] RBAC: Authorizing `identity:get_project(project_id=dabc91be3a7240e8bab6128f948854f0)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:29.321170 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644c717-fc95-44fb-b0f7-98a64d7b6e18 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:29.322847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 58/113] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:29 2022] GET /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:29.323653 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644c717-fc95-44fb-b0f7-98a64d7b6e18 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:29.342381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:29.353188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8b1bc-c9c0-49c4-b83a-93d998ca48c8 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:29.361913 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:29.388449 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fe670-6f2f-4ff6-9f6f-557be3859f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:29.389064 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fe670-6f2f-4ff6-9f6f-557be3859f92 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:29.389347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fe670-6f2f-4ff6-9f6f-557be3859f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:29.389623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fe670-6f2f-4ff6-9f6f-557be3859f92 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:29.390263 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080fe670-6f2f-4ff6-9f6f-557be3859f92 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:29.391733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080fe670-6f2f-4ff6-9f6f-557be3859f92 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:29.419643 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 56/114] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Oct 14 02:11:29 2022] POST /identity/v3/projects => generated 302 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:29.420474 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fe670-6f2f-4ff6-9f6f-557be3859f92 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:29.425774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644c717-fc95-44fb-b0f7-98a64d7b6e18 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:29.465300 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4403527-02d4-4dde-9ccc-ec4a4b52c0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:29.465869 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4403527-02d4-4dde-9ccc-ec4a4b52c0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:29.466160 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4403527-02d4-4dde-9ccc-ec4a4b52c0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:29.466445 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4403527-02d4-4dde-9ccc-ec4a4b52c0e7 None admin] PATH_INFO: `/v3/users/e989ab80fa194e5ca4a4e53f88ae8ecb` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:29.467668 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4403527-02d4-4dde-9ccc-ec4a4b52c0e7 None admin] RBAC: Authorizing `identity:get_user(user_id=e989ab80fa194e5ca4a4e53f88ae8ecb)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:29.469146 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4403527-02d4-4dde-9ccc-ec4a4b52c0e7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:29.475132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 59/115] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:29 2022] GET /identity/v3/users/e989ab80fa194e5ca4a4e53f88ae8ecb => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:29.475132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4403527-02d4-4dde-9ccc-ec4a4b52c0e7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:29.549173 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fe670-6f2f-4ff6-9f6f-557be3859f92 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:29.571684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:29.586946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b929e9-bfe8-420e-a614-c50295a937c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:29.587601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b929e9-bfe8-420e-a614-c50295a937c8 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:29.587875 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b929e9-bfe8-420e-a614-c50295a937c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:29.588148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b929e9-bfe8-420e-a614-c50295a937c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:29.588822 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b929e9-bfe8-420e-a614-c50295a937c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:29.589151 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b929e9-bfe8-420e-a614-c50295a937c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:29.590514 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b929e9-bfe8-420e-a614-c50295a937c8 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:29.590623 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4403527-02d4-4dde-9ccc-ec4a4b52c0e7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:29.600392 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 57/116] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:29.601213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b929e9-bfe8-420e-a614-c50295a937c8 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:29.629851 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee04dc3-1241-4432-962d-0b56069e7fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:29.630506 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee04dc3-1241-4432-962d-0b56069e7fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:29.630785 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee04dc3-1241-4432-962d-0b56069e7fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:29.631097 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee04dc3-1241-4432-962d-0b56069e7fd0 None admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:29.632314 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee04dc3-1241-4432-962d-0b56069e7fd0 None admin] RBAC: Authorizing `identity:get_project(project_id=dabc91be3a7240e8bab6128f948854f0)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:29.633779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee04dc3-1241-4432-962d-0b56069e7fd0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:29.634876 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:29.636334 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 60/117] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:29 2022] GET /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:29.636334 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee04dc3-1241-4432-962d-0b56069e7fd0 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:29.721846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b929e9-bfe8-420e-a614-c50295a937c8 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:29.748502 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee04dc3-1241-4432-962d-0b56069e7fd0 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:29.762744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61169050-9c71-4c9a-9a39-f3cb1299c309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:29.763418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61169050-9c71-4c9a-9a39-f3cb1299c309 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:29.763705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61169050-9c71-4c9a-9a39-f3cb1299c309 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:29.763983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61169050-9c71-4c9a-9a39-f3cb1299c309 None admin] PATH_INFO: `/v3/users/d8e2f2bcc0db47569a724614acdaccad` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:29.779184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61169050-9c71-4c9a-9a39-f3cb1299c309 None admin] RBAC: Authorizing `identity:get_user(user_id=d8e2f2bcc0db47569a724614acdaccad)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:29.780689 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61169050-9c71-4c9a-9a39-f3cb1299c309 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:29.782553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 58/118] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:29 2022] GET /identity/v3/users/d8e2f2bcc0db47569a724614acdaccad => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:29.783297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61169050-9c71-4c9a-9a39-f3cb1299c309 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:29.789989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb9e5b-ec17-459f-a240-f33149dd9609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:29.790865 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb9e5b-ec17-459f-a240-f33149dd9609 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:29.791397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb9e5b-ec17-459f-a240-f33149dd9609 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:29.791679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb9e5b-ec17-459f-a240-f33149dd9609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:29.792431 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb9e5b-ec17-459f-a240-f33149dd9609 None admin] RBAC: Adding query filter params (role.id=e460c7cf7b5b4c35a5a0e326f69a9fe3, scope.project.id=dabc91be3a7240e8bab6128f948854f0, user.id=e989ab80fa194e5ca4a4e53f88ae8ecb) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:29.792744 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb9e5b-ec17-459f-a240-f33149dd9609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:29.794295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb9e5b-ec17-459f-a240-f33149dd9609 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:29.801748 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 61/119] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:11:29 2022] GET /identity/v3/role_assignments?user.id=e989ab80fa194e5ca4a4e53f88ae8ecb&scope.project.id=dabc91be3a7240e8bab6128f948854f0&role.id=e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:29.803958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb9e5b-ec17-459f-a240-f33149dd9609 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:29.825022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:29.903116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:29.917774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb9e5b-ec17-459f-a240-f33149dd9609 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:29.928588 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61169050-9c71-4c9a-9a39-f3cb1299c309 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:29.954438 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12469f2c-3e98-47f1-a5d0-9d961148e4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:29.955119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12469f2c-3e98-47f1-a5d0-9d961148e4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:29.955411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12469f2c-3e98-47f1-a5d0-9d961148e4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:29.955702 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12469f2c-3e98-47f1-a5d0-9d961148e4a3 None admin] PATH_INFO: `/v3/users/e989ab80fa194e5ca4a4e53f88ae8ecb` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:29.957585 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12469f2c-3e98-47f1-a5d0-9d961148e4a3 None admin] RBAC: Authorizing `identity:get_user(user_id=e989ab80fa194e5ca4a4e53f88ae8ecb)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:29.959822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12469f2c-3e98-47f1-a5d0-9d961148e4a3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:29.962087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 62/120] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:29 2022] GET /identity/v3/users/e989ab80fa194e5ca4a4e53f88ae8ecb => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:29.963033 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12469f2c-3e98-47f1-a5d0-9d961148e4a3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:29.975433 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1861e2-736f-461d-b5ee-ca37faa32e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:29.976760 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1861e2-736f-461d-b5ee-ca37faa32e05 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:29.977228 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1861e2-736f-461d-b5ee-ca37faa32e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:29.977669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1861e2-736f-461d-b5ee-ca37faa32e05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:30.066553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1861e2-736f-461d-b5ee-ca37faa32e05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:30.068179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1861e2-736f-461d-b5ee-ca37faa32e05 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:30.078304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:30.092564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12469f2c-3e98-47f1-a5d0-9d961148e4a3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:30.131413 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf8dd0-34a4-45e8-94d0-aafb262fc3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:30.132038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf8dd0-34a4-45e8-94d0-aafb262fc3fd None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:30.132326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf8dd0-34a4-45e8-94d0-aafb262fc3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:30.132604 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf8dd0-34a4-45e8-94d0-aafb262fc3fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:30.133346 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf8dd0-34a4-45e8-94d0-aafb262fc3fd None admin] RBAC: Adding query filter params (role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b, scope.project.id=dabc91be3a7240e8bab6128f948854f0, user.id=6f436f18778d45bb9d581131675fcbb4) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:30.133656 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf8dd0-34a4-45e8-94d0-aafb262fc3fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:30.135128 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf8dd0-34a4-45e8-94d0-aafb262fc3fd None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:30.147113 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 63/121] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:11:29 2022] GET /identity/v3/role_assignments?user.id=6f436f18778d45bb9d581131675fcbb4&scope.project.id=dabc91be3a7240e8bab6128f948854f0&role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:30.147113 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf8dd0-34a4-45e8-94d0-aafb262fc3fd None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:30.169095 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:30.200054 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 59/122] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:11:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:11:30.202296 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1861e2-736f-461d-b5ee-ca37faa32e05 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:30.290941 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf8dd0-34a4-45e8-94d0-aafb262fc3fd None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:30.327754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1861e2-736f-461d-b5ee-ca37faa32e05 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:30.340362 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d084e4-7156-4daf-9248-48bc5e74c1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:30.341021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d084e4-7156-4daf-9248-48bc5e74c1ec None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:30.341306 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d084e4-7156-4daf-9248-48bc5e74c1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:30.341580 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d084e4-7156-4daf-9248-48bc5e74c1ec None admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:30.342736 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d084e4-7156-4daf-9248-48bc5e74c1ec None admin] RBAC: Authorizing `identity:get_project(project_id=dabc91be3a7240e8bab6128f948854f0)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:30.344493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d084e4-7156-4daf-9248-48bc5e74c1ec None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:30.346421 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 64/123] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:30 2022] GET /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:30.346684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d084e4-7156-4daf-9248-48bc5e74c1ec None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:30.373121 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:30.379977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89973046-c089-4dd7-a9e3-9dbcf84eaec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:30.380986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89973046-c089-4dd7-a9e3-9dbcf84eaec3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:30.380986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89973046-c089-4dd7-a9e3-9dbcf84eaec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:30.381172 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89973046-c089-4dd7-a9e3-9dbcf84eaec3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:30.441196 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:30.463888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89973046-c089-4dd7-a9e3-9dbcf84eaec3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:30.464846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89973046-c089-4dd7-a9e3-9dbcf84eaec3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:30.472817 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d084e4-7156-4daf-9248-48bc5e74c1ec None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:30.509129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2252652-1331-403d-98e4-79d940c9a62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:30.510011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252652-1331-403d-98e4-79d940c9a62f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:30.510472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252652-1331-403d-98e4-79d940c9a62f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:30.510922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252652-1331-403d-98e4-79d940c9a62f None admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:30.512315 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2252652-1331-403d-98e4-79d940c9a62f None admin] RBAC: Authorizing `identity:get_project(project_id=dabc91be3a7240e8bab6128f948854f0)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:30.513829 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2252652-1331-403d-98e4-79d940c9a62f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:30.515511 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 65/124] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:30 2022] GET /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:30.516343 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2252652-1331-403d-98e4-79d940c9a62f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:30.567593 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 60/125] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:11:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:11:30.568793 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89973046-c089-4dd7-a9e3-9dbcf84eaec3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:30.610011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:30.628317 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2252652-1331-403d-98e4-79d940c9a62f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:30.650376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:30.667248 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6d53b-ab76-448a-bf1e-70963f69c286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:30.667923 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6d53b-ab76-448a-bf1e-70963f69c286 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:30.668208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6d53b-ab76-448a-bf1e-70963f69c286 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:30.668487 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6d53b-ab76-448a-bf1e-70963f69c286 None admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0/users/d8e2f2bcc0db47569a724614acdaccad/roles/062b4d142bbc4085acd6efc13e6b2b74` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:30.670393 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6d53b-ab76-448a-bf1e-70963f69c286 None admin] RBAC: Authorizing `identity:create_grant(project_id=dabc91be3a7240e8bab6128f948854f0, user_id=d8e2f2bcc0db47569a724614acdaccad, role_id=062b4d142bbc4085acd6efc13e6b2b74)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:30.671852 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6d53b-ab76-448a-bf1e-70963f69c286 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:30.673286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89973046-c089-4dd7-a9e3-9dbcf84eaec3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:30.686653 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 66/126] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:11:30 2022] PUT /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0/users/d8e2f2bcc0db47569a724614acdaccad/roles/062b4d142bbc4085acd6efc13e6b2b74 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:11:30.696666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6d53b-ab76-448a-bf1e-70963f69c286 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:30.741976 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915cd3b6-efe0-4acb-8cf7-87bea311693b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:30.743276 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cd3b6-efe0-4acb-8cf7-87bea311693b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:30.743741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cd3b6-efe0-4acb-8cf7-87bea311693b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:30.744158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cd3b6-efe0-4acb-8cf7-87bea311693b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:30.745375 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915cd3b6-efe0-4acb-8cf7-87bea311693b None admin] RBAC: Adding query filter params (role.id=a3287688078248ecb19a6d9c85839631, scope.project.id=dabc91be3a7240e8bab6128f948854f0, user.id=e989ab80fa194e5ca4a4e53f88ae8ecb) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:30.746464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915cd3b6-efe0-4acb-8cf7-87bea311693b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:30.749084 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915cd3b6-efe0-4acb-8cf7-87bea311693b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:30.765988 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 61/127] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:11:30 2022] GET /identity/v3/role_assignments?user.id=e989ab80fa194e5ca4a4e53f88ae8ecb&scope.project.id=dabc91be3a7240e8bab6128f948854f0&role.id=a3287688078248ecb19a6d9c85839631 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:30.843597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6d53b-ab76-448a-bf1e-70963f69c286 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:30.886231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da76a0-9434-41f2-a7ee-44d5f6ebeab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:30.887049 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da76a0-9434-41f2-a7ee-44d5f6ebeab3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:30.887461 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da76a0-9434-41f2-a7ee-44d5f6ebeab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:30.887837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da76a0-9434-41f2-a7ee-44d5f6ebeab3 None admin] PATH_INFO: `/v3/projects/d6ba3ca61bd54d71a217b379e39a90e5` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:30.890068 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da76a0-9434-41f2-a7ee-44d5f6ebeab3 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ba3ca61bd54d71a217b379e39a90e5)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:30.891705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da76a0-9434-41f2-a7ee-44d5f6ebeab3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:30.893733 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 67/128] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:30 2022] GET /identity/v3/projects/d6ba3ca61bd54d71a217b379e39a90e5 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:30.900210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915cd3b6-efe0-4acb-8cf7-87bea311693b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:31.013650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915cd3b6-efe0-4acb-8cf7-87bea311693b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:31.050059 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f647f6-5771-4578-8de8-c105ca950fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:31.051641 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f647f6-5771-4578-8de8-c105ca950fcf None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:31.052126 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f647f6-5771-4578-8de8-c105ca950fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:31.052617 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f647f6-5771-4578-8de8-c105ca950fcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:31.053583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f647f6-5771-4578-8de8-c105ca950fcf None admin] RBAC: Adding query filter params (role.id=e460c7cf7b5b4c35a5a0e326f69a9fe3, scope.project.id=d6ba3ca61bd54d71a217b379e39a90e5, user.id=e989ab80fa194e5ca4a4e53f88ae8ecb) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:31.054092 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f647f6-5771-4578-8de8-c105ca950fcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:31.055788 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f647f6-5771-4578-8de8-c105ca950fcf None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:31.063057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 62/129] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:11:30 2022] GET /identity/v3/role_assignments?user.id=e989ab80fa194e5ca4a4e53f88ae8ecb&scope.project.id=d6ba3ca61bd54d71a217b379e39a90e5&role.id=e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:31.673425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e28453-bac1-439e-8eb0-6d5a590f3c15 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:31.673940 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e28453-bac1-439e-8eb0-6d5a590f3c15 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:31.674020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e28453-bac1-439e-8eb0-6d5a590f3c15 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:31.675380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 68/130] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:31.680535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b4a2-79a3-476b-810a-9dcdfe67a283 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:31.680824 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b4a2-79a3-476b-810a-9dcdfe67a283 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:31.681101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b4a2-79a3-476b-810a-9dcdfe67a283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:31.699231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-26d8b4a2-79a3-476b-810a-9dcdfe67a283 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:31.700407 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-26d8b4a2-79a3-476b-810a-9dcdfe67a283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:31.702290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:31.768687 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 63/131] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:32.040645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e28453-bac1-439e-8eb0-6d5a590f3c15 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:32.056905 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:32.143857 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e28453-bac1-439e-8eb0-6d5a590f3c15 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:32.176281 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766e86b-0797-48d0-b3b7-34bcb5ec691d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:32.176916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766e86b-0797-48d0-b3b7-34bcb5ec691d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:32.177201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766e86b-0797-48d0-b3b7-34bcb5ec691d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:32.177477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766e86b-0797-48d0-b3b7-34bcb5ec691d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:32.178574 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766e86b-0797-48d0-b3b7-34bcb5ec691d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:32.180038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766e86b-0797-48d0-b3b7-34bcb5ec691d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:32.181593 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 69/132] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:32.187050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8b4a2-79a3-476b-810a-9dcdfe67a283 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:32.283179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8b4a2-79a3-476b-810a-9dcdfe67a283 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:32.315243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f41c3f-fe0f-45af-b5f7-cffaf29686a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:32.316302 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f41c3f-fe0f-45af-b5f7-cffaf29686a9 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:32.316591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f41c3f-fe0f-45af-b5f7-cffaf29686a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:32.316864 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f41c3f-fe0f-45af-b5f7-cffaf29686a9 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:32.317506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f41c3f-fe0f-45af-b5f7-cffaf29686a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:32.318948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f41c3f-fe0f-45af-b5f7-cffaf29686a9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:32.328740 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:32.339647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 64/133] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:11:32 2022] POST /identity/v3/users => generated 283 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:32.883330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a13e1a-b4a8-4314-9877-06bdf369d45c None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:32.884465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a13e1a-b4a8-4314-9877-06bdf369d45c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:32.884990 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a13e1a-b4a8-4314-9877-06bdf369d45c None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:32.886578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 70/134] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:32.892102 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccaed7-1662-448a-9980-06c816693a03 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:32.892386 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccaed7-1662-448a-9980-06c816693a03 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:32.892666 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccaed7-1662-448a-9980-06c816693a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:32.911499 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-feccaed7-1662-448a-9980-06c816693a03 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:32.912680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-feccaed7-1662-448a-9980-06c816693a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:32.980467 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 65/135] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:33.249170 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a13e1a-b4a8-4314-9877-06bdf369d45c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:33.341324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:33.361477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a13e1a-b4a8-4314-9877-06bdf369d45c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:33.397156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc349da-17da-4890-abc3-e9097df51fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:33.397811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc349da-17da-4890-abc3-e9097df51fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:33.398097 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc349da-17da-4890-abc3-e9097df51fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:33.398380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc349da-17da-4890-abc3-e9097df51fe6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:33.410460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc349da-17da-4890-abc3-e9097df51fe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-7bc349da-17da-4890-abc3-e9097df51fe6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:33.415379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:33.416192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:33.417582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:33.417582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:33.417582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:33.417582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:33.417582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:33.417582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:33.417582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:33.417582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:33.417582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:33.417582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:33.417582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:11:33.417582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:33.417582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 71/136] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:33.422479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccaed7-1662-448a-9980-06c816693a03 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:33.527836 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccaed7-1662-448a-9980-06c816693a03 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:33.553377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:33.564717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507447b-51da-4228-8d41-dea85ed9a172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:33.565335 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507447b-51da-4228-8d41-dea85ed9a172 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:33.565616 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507447b-51da-4228-8d41-dea85ed9a172 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:33.566035 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507447b-51da-4228-8d41-dea85ed9a172 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:33.566762 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507447b-51da-4228-8d41-dea85ed9a172 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:33.567282 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507447b-51da-4228-8d41-dea85ed9a172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:33.568971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507447b-51da-4228-8d41-dea85ed9a172 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:33.577634 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 66/137] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:33.583906 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc349da-17da-4890-abc3-e9097df51fe6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:33.685155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc349da-17da-4890-abc3-e9097df51fe6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:33.718686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d855ca-dfbc-4bc2-85c3-888963aa1d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:33.719402 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d855ca-dfbc-4bc2-85c3-888963aa1d00 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:33.719684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d855ca-dfbc-4bc2-85c3-888963aa1d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:33.720012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d855ca-dfbc-4bc2-85c3-888963aa1d00 None admin] PATH_INFO: `/v3/users/d8e2f2bcc0db47569a724614acdaccad` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:33.721142 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d855ca-dfbc-4bc2-85c3-888963aa1d00 None admin] RBAC: Authorizing `identity:get_user(user_id=d8e2f2bcc0db47569a724614acdaccad)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:33.722634 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d855ca-dfbc-4bc2-85c3-888963aa1d00 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:33.724325 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 72/138] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:33 2022] GET /identity/v3/users/d8e2f2bcc0db47569a724614acdaccad => generated 289 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:33.731059 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507447b-51da-4228-8d41-dea85ed9a172 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:33.836045 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507447b-51da-4228-8d41-dea85ed9a172 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:33.874943 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcb840-f13d-49d9-af3f-2140369b74b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:33.875588 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcb840-f13d-49d9-af3f-2140369b74b9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:33.875867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcb840-f13d-49d9-af3f-2140369b74b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:33.876139 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcb840-f13d-49d9-af3f-2140369b74b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:33.933483 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:33.947411 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcb840-f13d-49d9-af3f-2140369b74b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:33.948905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcb840-f13d-49d9-af3f-2140369b74b9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:34.049333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 67/139] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:11:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 14 02:11:34.055614 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d855ca-dfbc-4bc2-85c3-888963aa1d00 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:34.068496 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:34.158624 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d855ca-dfbc-4bc2-85c3-888963aa1d00 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:34.193910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041f2da-94ac-4af1-a9f3-1e860a6476ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:34.194560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041f2da-94ac-4af1-a9f3-1e860a6476ad None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:34.194846 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041f2da-94ac-4af1-a9f3-1e860a6476ad None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:34.195151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041f2da-94ac-4af1-a9f3-1e860a6476ad None admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:34.196367 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041f2da-94ac-4af1-a9f3-1e860a6476ad None admin] RBAC: Authorizing `identity:get_project(project_id=dabc91be3a7240e8bab6128f948854f0)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:34.197813 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041f2da-94ac-4af1-a9f3-1e860a6476ad None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:34.199476 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 73/140] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:34 2022] GET /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:34.205331 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcb840-f13d-49d9-af3f-2140369b74b9 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:34.293692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:34.310034 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcb840-f13d-49d9-af3f-2140369b74b9 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:34.345197 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545d1c3-537c-4a88-a339-3f8b633b9bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:34.345817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545d1c3-537c-4a88-a339-3f8b633b9bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:34.346103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545d1c3-537c-4a88-a339-3f8b633b9bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:34.346382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545d1c3-537c-4a88-a339-3f8b633b9bb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:34.347252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545d1c3-537c-4a88-a339-3f8b633b9bb7 None admin] RBAC: Adding query filter params (role.id=062b4d142bbc4085acd6efc13e6b2b74, scope.project.id=dabc91be3a7240e8bab6128f948854f0, user.id=d8e2f2bcc0db47569a724614acdaccad) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:34.347570 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545d1c3-537c-4a88-a339-3f8b633b9bb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:34.349501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545d1c3-537c-4a88-a339-3f8b633b9bb7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:34.355918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 68/141] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:11:34 2022] GET /identity/v3/role_assignments?user.id=d8e2f2bcc0db47569a724614acdaccad&scope.project.id=dabc91be3a7240e8bab6128f948854f0&role.id=062b4d142bbc4085acd6efc13e6b2b74 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:34.692776 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845d45a-3eaa-42da-b2dd-5f524f0c400c None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:34.693126 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845d45a-3eaa-42da-b2dd-5f524f0c400c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:34.693328 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845d45a-3eaa-42da-b2dd-5f524f0c400c None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:34.694907 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 74/142] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:34.700151 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cd1fe-9091-42f3-af7f-5dfdb0925a2c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:34.700665 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cd1fe-9091-42f3-af7f-5dfdb0925a2c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:34.700825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cd1fe-9091-42f3-af7f-5dfdb0925a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:34.722583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2a0cd1fe-9091-42f3-af7f-5dfdb0925a2c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:34.723951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2a0cd1fe-9091-42f3-af7f-5dfdb0925a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:34.798133 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 69/143] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:34.830005 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5e38f-18ca-4ce0-bb65-260ed1b3bd7d None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:34.830333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5e38f-18ca-4ce0-bb65-260ed1b3bd7d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:34.830608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5e38f-18ca-4ce0-bb65-260ed1b3bd7d None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:34.833176 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 75/144] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:34.838908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604e1a5-ea30-4d2d-b7dc-e95de83a0c4a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:34.839256 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604e1a5-ea30-4d2d-b7dc-e95de83a0c4a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:34.839529 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604e1a5-ea30-4d2d-b7dc-e95de83a0c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:34.863365 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a604e1a5-ea30-4d2d-b7dc-e95de83a0c4a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:34.864921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a604e1a5-ea30-4d2d-b7dc-e95de83a0c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:34.943334 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 70/145] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:34.963547 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81685567-eaf3-4079-b1dc-0f243eea1862 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:34.963833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81685567-eaf3-4079-b1dc-0f243eea1862 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:34.965489 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81685567-eaf3-4079-b1dc-0f243eea1862 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:34.965602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 76/146] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:34.971205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df19d9b-87e7-419b-b857-5b4927f64bac None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:34.971501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df19d9b-87e7-419b-b857-5b4927f64bac None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:34.971782 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df19d9b-87e7-419b-b857-5b4927f64bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:34.993214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4df19d9b-87e7-419b-b857-5b4927f64bac None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:34.994402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4df19d9b-87e7-419b-b857-5b4927f64bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:34.997505 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:35.068367 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 71/147] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:35.076339 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81685567-eaf3-4079-b1dc-0f243eea1862 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:35.154503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:35.178648 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81685567-eaf3-4079-b1dc-0f243eea1862 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:35.211032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54907e7c-7735-44d4-8f44-b4281ebda5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:35.211629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54907e7c-7735-44d4-8f44-b4281ebda5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:35.211958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54907e7c-7735-44d4-8f44-b4281ebda5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:35.212228 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54907e7c-7735-44d4-8f44-b4281ebda5b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:35.221752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df19d9b-87e7-419b-b857-5b4927f64bac None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:35.223124 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54907e7c-7735-44d4-8f44-b4281ebda5b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-54907e7c-7735-44d4-8f44-b4281ebda5b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:35.228522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:35.229371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:35.230321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:35.230321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:35.230321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:35.230321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:35.230321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:35.230321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:35.230321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:35.230321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:35.230321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:35.230321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:35.230321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:11:35.230321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:35.230321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 77/148] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:11:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:35.236837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54907e7c-7735-44d4-8f44-b4281ebda5b3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:35.329666 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df19d9b-87e7-419b-b857-5b4927f64bac None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:35.342191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54907e7c-7735-44d4-8f44-b4281ebda5b3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:35.364729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ba337-a613-4013-a8d9-468a502d60f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:35.365353 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ba337-a613-4013-a8d9-468a502d60f3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:35.365625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ba337-a613-4013-a8d9-468a502d60f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:35.365889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ba337-a613-4013-a8d9-468a502d60f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:35.373335 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:35.376606 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89c151-5171-422c-b208-93f51f8006d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:35.377138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89c151-5171-422c-b208-93f51f8006d0 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:35.377287 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ba337-a613-4013-a8d9-468a502d60f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:35.377411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89c151-5171-422c-b208-93f51f8006d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:35.377678 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89c151-5171-422c-b208-93f51f8006d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:35.378372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89c151-5171-422c-b208-93f51f8006d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:35.378670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89c151-5171-422c-b208-93f51f8006d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:35.380053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89c151-5171-422c-b208-93f51f8006d0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-be1ba337-a613-4013-a8d9-468a502d60f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:35.382114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:35.382957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:35.383995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:35.383995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:35.383995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:35.383995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:35.383995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:35.383995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:35.383995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:35.383995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:35.383995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:35.383995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:35.383995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:11:35.383995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:35.383995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 72/149] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:11:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:11:35.385299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:35.388089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ba337-a613-4013-a8d9-468a502d60f3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:35.391249 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 78/150] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:35.392070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89c151-5171-422c-b208-93f51f8006d0 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:35.488093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ba337-a613-4013-a8d9-468a502d60f3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:35.489170 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89c151-5171-422c-b208-93f51f8006d0 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:35.521872 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8aed1-bfe9-47c7-8c9e-ffa0cf53d209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:35.522505 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8aed1-bfe9-47c7-8c9e-ffa0cf53d209 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:35.522790 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8aed1-bfe9-47c7-8c9e-ffa0cf53d209 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:35.522933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15c878-e809-46d3-9b24-16f2013eb308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:35.523164 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8aed1-bfe9-47c7-8c9e-ffa0cf53d209 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:35.523428 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15c878-e809-46d3-9b24-16f2013eb308 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:35.523708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15c878-e809-46d3-9b24-16f2013eb308 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:35.523981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15c878-e809-46d3-9b24-16f2013eb308 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:35.524222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8aed1-bfe9-47c7-8c9e-ffa0cf53d209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:35.524643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15c878-e809-46d3-9b24-16f2013eb308 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:35.524947 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15c878-e809-46d3-9b24-16f2013eb308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:35.525682 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8aed1-bfe9-47c7-8c9e-ffa0cf53d209 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:35.526307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15c878-e809-46d3-9b24-16f2013eb308 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:35.527277 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 73/151] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:35.529139 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8aed1-bfe9-47c7-8c9e-ffa0cf53d209 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:35.535968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 79/152] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:11:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:35.536758 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15c878-e809-46d3-9b24-16f2013eb308 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:35.593638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:35.596133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:35.628710 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8aed1-bfe9-47c7-8c9e-ffa0cf53d209 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:35.633760 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15c878-e809-46d3-9b24-16f2013eb308 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:35.661988 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d08d3-3f5e-4c65-97eb-57cc51b107d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:35.662706 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d08d3-3f5e-4c65-97eb-57cc51b107d3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:35.662989 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d08d3-3f5e-4c65-97eb-57cc51b107d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:35.663378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d08d3-3f5e-4c65-97eb-57cc51b107d3 None admin] PATH_INFO: `/v3/users/3f7355d66d79499ba72d14c2a665630a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:35.666729 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a72674-6980-42bf-b570-4d1c5e16b006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:35.667356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72674-6980-42bf-b570-4d1c5e16b006 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:35.667637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72674-6980-42bf-b570-4d1c5e16b006 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:35.667909 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72674-6980-42bf-b570-4d1c5e16b006 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:35.668551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a72674-6980-42bf-b570-4d1c5e16b006 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:35.669989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a72674-6980-42bf-b570-4d1c5e16b006 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:35.676204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d08d3-3f5e-4c65-97eb-57cc51b107d3 None admin] RBAC: Authorizing `identity:get_user(user_id=3f7355d66d79499ba72d14c2a665630a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:35.677645 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d08d3-3f5e-4c65-97eb-57cc51b107d3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:35.679486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 74/153] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:35 2022] GET /identity/v3/users/3f7355d66d79499ba72d14c2a665630a => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:35.680180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d08d3-3f5e-4c65-97eb-57cc51b107d3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:35.703094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 80/154] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:11:35 2022] POST /identity/v3/users => generated 297 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:35.703094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a72674-6980-42bf-b570-4d1c5e16b006 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:35.799039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d08d3-3f5e-4c65-97eb-57cc51b107d3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:35.826908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a72674-6980-42bf-b570-4d1c5e16b006 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:35.827613 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:35.840017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661ace7-0a6c-4636-a80e-bd729835b19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:35.840970 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661ace7-0a6c-4636-a80e-bd729835b19d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:35.841254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661ace7-0a6c-4636-a80e-bd729835b19d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:35.841530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661ace7-0a6c-4636-a80e-bd729835b19d None admin] PATH_INFO: `/v3/users/3f7355d66d79499ba72d14c2a665630a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:35.843335 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661ace7-0a6c-4636-a80e-bd729835b19d None admin] RBAC: Authorizing `identity:get_user(user_id=3f7355d66d79499ba72d14c2a665630a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:35.845144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661ace7-0a6c-4636-a80e-bd729835b19d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:35.846424 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:35.853602 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 75/155] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:35 2022] GET /identity/v3/users/3f7355d66d79499ba72d14c2a665630a => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:35.854575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661ace7-0a6c-4636-a80e-bd729835b19d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:35.870315 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5e456-da24-4ace-b63c-f13d8ae07ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:35.870976 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5e456-da24-4ace-b63c-f13d8ae07ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:35.871290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5e456-da24-4ace-b63c-f13d8ae07ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:35.871561 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5e456-da24-4ace-b63c-f13d8ae07ba1 None admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:35.872805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd5e456-da24-4ace-b63c-f13d8ae07ba1 None admin] RBAC: Authorizing `identity:get_project(project_id=f94fce41714c4f3099c1ff9eb62caf49)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:35.874525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd5e456-da24-4ace-b63c-f13d8ae07ba1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:35.881529 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 81/156] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:35 2022] GET /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:35.882532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5e456-da24-4ace-b63c-f13d8ae07ba1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:35.966934 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661ace7-0a6c-4636-a80e-bd729835b19d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:36.009416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5e456-da24-4ace-b63c-f13d8ae07ba1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:36.010891 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7115b70-47b5-4a28-8e64-c681d35b52e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:36.011754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7115b70-47b5-4a28-8e64-c681d35b52e1 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:36.012869 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7115b70-47b5-4a28-8e64-c681d35b52e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:36.013167 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7115b70-47b5-4a28-8e64-c681d35b52e1 None admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:36.014301 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7115b70-47b5-4a28-8e64-c681d35b52e1 None admin] RBAC: Authorizing `identity:get_project(project_id=f94fce41714c4f3099c1ff9eb62caf49)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:36.016412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7115b70-47b5-4a28-8e64-c681d35b52e1 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:36.019395 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 76/157] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:35 2022] GET /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:36.025053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7115b70-47b5-4a28-8e64-c681d35b52e1 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:36.052594 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980dfb6-4186-4c4f-8cc9-67cbe2413fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:36.053187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980dfb6-4186-4c4f-8cc9-67cbe2413fe4 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:36.053464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980dfb6-4186-4c4f-8cc9-67cbe2413fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:36.053737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980dfb6-4186-4c4f-8cc9-67cbe2413fe4 None admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49/users/3f7355d66d79499ba72d14c2a665630a/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:36.057554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980dfb6-4186-4c4f-8cc9-67cbe2413fe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f94fce41714c4f3099c1ff9eb62caf49, user_id=3f7355d66d79499ba72d14c2a665630a, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:36.059006 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980dfb6-4186-4c4f-8cc9-67cbe2413fe4 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:36.076943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 82/158] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:11:35 2022] PUT /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49/users/3f7355d66d79499ba72d14c2a665630a/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:11:36.099680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:36.156704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7115b70-47b5-4a28-8e64-c681d35b52e1 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:36.197190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96c3b2-c02f-4911-9247-bd1f01391465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:36.198283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96c3b2-c02f-4911-9247-bd1f01391465 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:36.198753 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96c3b2-c02f-4911-9247-bd1f01391465 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:36.199250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96c3b2-c02f-4911-9247-bd1f01391465 None admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49/users/3f7355d66d79499ba72d14c2a665630a/roles/a3287688078248ecb19a6d9c85839631` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:36.201662 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96c3b2-c02f-4911-9247-bd1f01391465 None admin] RBAC: Authorizing `identity:create_grant(project_id=f94fce41714c4f3099c1ff9eb62caf49, user_id=3f7355d66d79499ba72d14c2a665630a, role_id=a3287688078248ecb19a6d9c85839631)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:36.203384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96c3b2-c02f-4911-9247-bd1f01391465 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:36.222961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 77/159] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:11:36 2022] PUT /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49/users/3f7355d66d79499ba72d14c2a665630a/roles/a3287688078248ecb19a6d9c85839631 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:11:37.991295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baff6cb-2d40-4c49-9d63-be76d3e6d8aa None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:37.991801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baff6cb-2d40-4c49-9d63-be76d3e6d8aa None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:37.991801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baff6cb-2d40-4c49-9d63-be76d3e6d8aa None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:37.993133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 83/160] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:37.999401 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea281f45-fef9-4245-a15b-0fbde8c6ceaa None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:37.999914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea281f45-fef9-4245-a15b-0fbde8c6ceaa None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:37.999914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea281f45-fef9-4245-a15b-0fbde8c6ceaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:38.020002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ea281f45-fef9-4245-a15b-0fbde8c6ceaa None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:38.021189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ea281f45-fef9-4245-a15b-0fbde8c6ceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:38.094485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 78/161] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:11:38.126090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d90c0-4448-425c-8643-7750a0e06904 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:38.126708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d90c0-4448-425c-8643-7750a0e06904 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:38.127235 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d90c0-4448-425c-8643-7750a0e06904 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:38.130111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 84/162] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:38.135647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005d696-9fa8-4099-ac68-ce57f0b54e3e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:38.136183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005d696-9fa8-4099-ac68-ce57f0b54e3e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:38.136654 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005d696-9fa8-4099-ac68-ce57f0b54e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:38.155263 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6005d696-9fa8-4099-ac68-ce57f0b54e3e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:38.156759 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6005d696-9fa8-4099-ac68-ce57f0b54e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:38.206694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:38.230565 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 79/163] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:38.351133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308a799-aec6-43a9-9a06-4e5fb2465083 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:38.351797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308a799-aec6-43a9-9a06-4e5fb2465083 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:38.352283 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308a799-aec6-43a9-9a06-4e5fb2465083 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:38.354093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 85/164] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:38.359559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793815-a9f8-4210-92fb-a63afb22f481 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:38.360095 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793815-a9f8-4210-92fb-a63afb22f481 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:38.360598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793815-a9f8-4210-92fb-a63afb22f481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:38.369410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308a799-aec6-43a9-9a06-4e5fb2465083 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:38.382819 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0d793815-a9f8-4210-92fb-a63afb22f481 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:38.384517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0d793815-a9f8-4210-92fb-a63afb22f481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:38.426705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:38.466205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 80/165] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:38.467830 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de2b3f-23f1-46dd-a8fe-d607e213301b None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:38.468107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de2b3f-23f1-46dd-a8fe-d607e213301b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:38.468376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de2b3f-23f1-46dd-a8fe-d607e213301b None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:38.469780 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 81/166] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:38.475368 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061effe9-2ce7-4ae5-a786-d20e0eeac6ee None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:38.475645 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061effe9-2ce7-4ae5-a786-d20e0eeac6ee None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:38.475916 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061effe9-2ce7-4ae5-a786-d20e0eeac6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:38.494934 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-061effe9-2ce7-4ae5-a786-d20e0eeac6ee None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:38.496139 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-061effe9-2ce7-4ae5-a786-d20e0eeac6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:38.499291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308a799-aec6-43a9-9a06-4e5fb2465083 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:38.534198 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e887c-928c-48f7-a6a0-bb56b2518ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:38.535694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e887c-928c-48f7-a6a0-bb56b2518ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:38.535973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e887c-928c-48f7-a6a0-bb56b2518ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:38.536243 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e887c-928c-48f7-a6a0-bb56b2518ec7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:38.546080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e887c-928c-48f7-a6a0-bb56b2518ec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-064e887c-928c-48f7-a6a0-bb56b2518ec7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:38.551454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:38.552305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:38.553056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:38.553056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:38.553056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:38.553056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:38.553056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:38.553056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:38.553056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:38.553056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:38.553056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:38.553056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:38.553056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:38.553056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:38.553056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 86/167] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:11:38.553681 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e887c-928c-48f7-a6a0-bb56b2518ec7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:38.572888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 82/168] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:11:38.574073 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061effe9-2ce7-4ae5-a786-d20e0eeac6ee None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:38.637372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:38.671615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e887c-928c-48f7-a6a0-bb56b2518ec7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:38.680218 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:38.714311 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3024d-f39d-4ec2-b2c5-b551ed093f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:38.715039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3024d-f39d-4ec2-b2c5-b551ed093f4f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:38.715433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3024d-f39d-4ec2-b2c5-b551ed093f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:38.715769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3024d-f39d-4ec2-b2c5-b551ed093f4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:38.716038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061effe9-2ce7-4ae5-a786-d20e0eeac6ee None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:38.716997 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3024d-f39d-4ec2-b2c5-b551ed093f4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:38.718704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3024d-f39d-4ec2-b2c5-b551ed093f4f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:38.725499 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 87/169] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:38.728788 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3024d-f39d-4ec2-b2c5-b551ed093f4f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:38.765365 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178f3af-a61f-4bee-bb72-b4135ad96d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:38.765973 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178f3af-a61f-4bee-bb72-b4135ad96d16 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:38.766247 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178f3af-a61f-4bee-bb72-b4135ad96d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:38.766514 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178f3af-a61f-4bee-bb72-b4135ad96d16 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:38.767199 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178f3af-a61f-4bee-bb72-b4135ad96d16 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:38.767498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178f3af-a61f-4bee-bb72-b4135ad96d16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:38.769593 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178f3af-a61f-4bee-bb72-b4135ad96d16 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:38.777812 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 83/170] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:38.782804 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178f3af-a61f-4bee-bb72-b4135ad96d16 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:38.840011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3024d-f39d-4ec2-b2c5-b551ed093f4f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:38.873930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94213c-e0aa-4594-8e9a-8dc108493462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:38.874539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94213c-e0aa-4594-8e9a-8dc108493462 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:38.874819 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94213c-e0aa-4594-8e9a-8dc108493462 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:38.875112 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94213c-e0aa-4594-8e9a-8dc108493462 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:38.875764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94213c-e0aa-4594-8e9a-8dc108493462 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:38.877203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94213c-e0aa-4594-8e9a-8dc108493462 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:38.884321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178f3af-a61f-4bee-bb72-b4135ad96d16 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:38.886936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:38.897199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 88/171] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:11:38 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:38.897949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94213c-e0aa-4594-8e9a-8dc108493462 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:38.929025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e10061-6639-49e2-94da-691720311cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:38.929771 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e10061-6639-49e2-94da-691720311cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:38.930103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e10061-6639-49e2-94da-691720311cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:38.930393 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e10061-6639-49e2-94da-691720311cb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:38.932743 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:38.943251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e10061-6639-49e2-94da-691720311cb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-25e10061-6639-49e2-94da-691720311cb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:38.948987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:38.949951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:38.951159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:38.951159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:38.951159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:38.951159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:38.951159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:38.951159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:38.951159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:38.951159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:38.951159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:38.951159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:38.951159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:11:38.951159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:38.951159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 84/172] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:11:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:38.951159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e10061-6639-49e2-94da-691720311cb2 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:39.017231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94213c-e0aa-4594-8e9a-8dc108493462 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:39.058157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cacbe-df85-4b83-bd3d-4f59cdf46f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:39.058819 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cacbe-df85-4b83-bd3d-4f59cdf46f5c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:39.059127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cacbe-df85-4b83-bd3d-4f59cdf46f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:39.059408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cacbe-df85-4b83-bd3d-4f59cdf46f5c None admin] PATH_INFO: `/v3/users/1d28c2dd35f44cd89b1159a51aba33f8` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:39.071959 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e10061-6639-49e2-94da-691720311cb2 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:39.079854 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cacbe-df85-4b83-bd3d-4f59cdf46f5c None admin] RBAC: Authorizing `identity:get_user(user_id=1d28c2dd35f44cd89b1159a51aba33f8)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:39.082138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cacbe-df85-4b83-bd3d-4f59cdf46f5c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:39.083873 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 89/173] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:38 2022] GET /identity/v3/users/1d28c2dd35f44cd89b1159a51aba33f8 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:39.089521 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cacbe-df85-4b83-bd3d-4f59cdf46f5c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:39.116213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa26f8b-d4ba-447c-8676-e6127778158f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:39.116881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa26f8b-d4ba-447c-8676-e6127778158f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:39.119486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa26f8b-d4ba-447c-8676-e6127778158f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:39.119486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa26f8b-d4ba-447c-8676-e6127778158f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:39.130000 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa26f8b-d4ba-447c-8676-e6127778158f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-2fa26f8b-d4ba-447c-8676-e6127778158f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:39.135366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:39.136215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:39.136982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:39.136982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:39.136982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:39.136982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:39.136982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:39.136982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:39.136982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:39.136982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:39.136982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:39.136982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:39.136982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:11:39.136982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:39.136982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 85/174] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:11:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:39.137603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa26f8b-d4ba-447c-8676-e6127778158f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:39.169315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:39.197907 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:39.212970 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cacbe-df85-4b83-bd3d-4f59cdf46f5c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:39.254257 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28d81d-a876-4eff-a6b2-215e67c75f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:39.254880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28d81d-a876-4eff-a6b2-215e67c75f9b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:39.255194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28d81d-a876-4eff-a6b2-215e67c75f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:39.255472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28d81d-a876-4eff-a6b2-215e67c75f9b None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:39.256133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28d81d-a876-4eff-a6b2-215e67c75f9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:39.256432 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28d81d-a876-4eff-a6b2-215e67c75f9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:39.257800 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28d81d-a876-4eff-a6b2-215e67c75f9b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:39.264003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa26f8b-d4ba-447c-8676-e6127778158f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:39.266962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 90/175] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:39.269373 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28d81d-a876-4eff-a6b2-215e67c75f9b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:39.317667 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bb3dd-e85e-4b51-97a9-d3365193c5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:39.319005 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bb3dd-e85e-4b51-97a9-d3365193c5f4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:39.319413 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bb3dd-e85e-4b51-97a9-d3365193c5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:39.319776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bb3dd-e85e-4b51-97a9-d3365193c5f4 None admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:39.321496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bb3dd-e85e-4b51-97a9-d3365193c5f4 None admin] RBAC: Authorizing `identity:get_project(project_id=f94fce41714c4f3099c1ff9eb62caf49)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:39.323716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bb3dd-e85e-4b51-97a9-d3365193c5f4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:39.326152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 86/176] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:39 2022] GET /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:39.327317 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bb3dd-e85e-4b51-97a9-d3365193c5f4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:39.385615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28d81d-a876-4eff-a6b2-215e67c75f9b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:39.429637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c153657-01ab-4141-921d-4d2a8ba3250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:39.430341 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c153657-01ab-4141-921d-4d2a8ba3250f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:39.431304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c153657-01ab-4141-921d-4d2a8ba3250f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:39.431696 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c153657-01ab-4141-921d-4d2a8ba3250f None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:39.432478 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c153657-01ab-4141-921d-4d2a8ba3250f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:39.432881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c153657-01ab-4141-921d-4d2a8ba3250f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:39.434888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c153657-01ab-4141-921d-4d2a8ba3250f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:39.451839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 91/177] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:11:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:39.451839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c153657-01ab-4141-921d-4d2a8ba3250f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:39.454947 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:39.465308 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bb3dd-e85e-4b51-97a9-d3365193c5f4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:39.469426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:39.502736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e7f8b-6d8b-4830-bd87-4c65774c9fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:39.504097 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e7f8b-6d8b-4830-bd87-4c65774c9fcb None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:39.504581 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e7f8b-6d8b-4830-bd87-4c65774c9fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:39.505127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e7f8b-6d8b-4830-bd87-4c65774c9fcb None admin] PATH_INFO: `/v3/users/3f7355d66d79499ba72d14c2a665630a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:39.506531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e7f8b-6d8b-4830-bd87-4c65774c9fcb None admin] RBAC: Authorizing `identity:get_user(user_id=3f7355d66d79499ba72d14c2a665630a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:39.508278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e7f8b-6d8b-4830-bd87-4c65774c9fcb None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:39.510162 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 87/178] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:39 2022] GET /identity/v3/users/3f7355d66d79499ba72d14c2a665630a => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:39.511213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e7f8b-6d8b-4830-bd87-4c65774c9fcb None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:39.591420 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c153657-01ab-4141-921d-4d2a8ba3250f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:39.622406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e7f8b-6d8b-4830-bd87-4c65774c9fcb None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:39.635703 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c8b14-6ab4-4bf5-b01e-74945ea84f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:39.636377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c8b14-6ab4-4bf5-b01e-74945ea84f06 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:39.636655 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c8b14-6ab4-4bf5-b01e-74945ea84f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:39.636930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c8b14-6ab4-4bf5-b01e-74945ea84f06 None admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49/users/1d28c2dd35f44cd89b1159a51aba33f8/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:39.639768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c8b14-6ab4-4bf5-b01e-74945ea84f06 None admin] RBAC: Authorizing `identity:create_grant(project_id=f94fce41714c4f3099c1ff9eb62caf49, user_id=1d28c2dd35f44cd89b1159a51aba33f8, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:39.641619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c8b14-6ab4-4bf5-b01e-74945ea84f06 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:39.662099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 92/179] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:11:39 2022] PUT /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49/users/1d28c2dd35f44cd89b1159a51aba33f8/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:11:39.662614 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09202474-6681-4a7e-b47f-7a7be3b54503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:39.662975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c8b14-6ab4-4bf5-b01e-74945ea84f06 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:39.663898 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09202474-6681-4a7e-b47f-7a7be3b54503 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:39.664391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09202474-6681-4a7e-b47f-7a7be3b54503 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:39.664887 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09202474-6681-4a7e-b47f-7a7be3b54503 None admin] PATH_INFO: `/v3/users/3f7355d66d79499ba72d14c2a665630a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:39.666489 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09202474-6681-4a7e-b47f-7a7be3b54503 None admin] RBAC: Authorizing `identity:get_user(user_id=3f7355d66d79499ba72d14c2a665630a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:39.668386 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09202474-6681-4a7e-b47f-7a7be3b54503 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:39.670406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 88/180] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:39 2022] GET /identity/v3/users/3f7355d66d79499ba72d14c2a665630a => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:39.676860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09202474-6681-4a7e-b47f-7a7be3b54503 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:39.742089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:39.757365 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:39.796160 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c8b14-6ab4-4bf5-b01e-74945ea84f06 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:39.837254 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012615d-8904-45a9-a854-33e0e6bf2775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:39.837897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012615d-8904-45a9-a854-33e0e6bf2775 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:39.838248 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012615d-8904-45a9-a854-33e0e6bf2775 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:39.838527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012615d-8904-45a9-a854-33e0e6bf2775 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:39.920594 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09202474-6681-4a7e-b47f-7a7be3b54503 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:39.925632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012615d-8904-45a9-a854-33e0e6bf2775 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:39.927295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012615d-8904-45a9-a854-33e0e6bf2775 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:39.983667 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc3ee2-29b5-49af-b7d2-dc9eeb923caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:39.986083 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc3ee2-29b5-49af-b7d2-dc9eeb923caf None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:39.986418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc3ee2-29b5-49af-b7d2-dc9eeb923caf None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:39.986696 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc3ee2-29b5-49af-b7d2-dc9eeb923caf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:40.053308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:40.090855 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 93/181] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:11:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:11:40.096510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012615d-8904-45a9-a854-33e0e6bf2775 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:40.115972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc3ee2-29b5-49af-b7d2-dc9eeb923caf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:40.117460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc3ee2-29b5-49af-b7d2-dc9eeb923caf None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:40.178233 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:40.273804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012615d-8904-45a9-a854-33e0e6bf2775 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:40.315753 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 89/182] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:11:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:11:40.321621 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc3ee2-29b5-49af-b7d2-dc9eeb923caf None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:40.334802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0660e5f-6f4b-46fe-89c3-839f1bb6656b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:40.335477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0660e5f-6f4b-46fe-89c3-839f1bb6656b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:40.335757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0660e5f-6f4b-46fe-89c3-839f1bb6656b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:40.336073 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0660e5f-6f4b-46fe-89c3-839f1bb6656b None admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:40.337203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0660e5f-6f4b-46fe-89c3-839f1bb6656b None admin] RBAC: Authorizing `identity:get_project(project_id=f94fce41714c4f3099c1ff9eb62caf49)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:40.338649 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0660e5f-6f4b-46fe-89c3-839f1bb6656b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:40.340531 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 94/183] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:40 2022] GET /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:40.347233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0660e5f-6f4b-46fe-89c3-839f1bb6656b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:40.441886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:40.449870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc3ee2-29b5-49af-b7d2-dc9eeb923caf None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:40.515450 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:40.524533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68ca45-2ff0-4d93-87ff-a0c9a4f2839b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:40.525102 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68ca45-2ff0-4d93-87ff-a0c9a4f2839b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:40.525379 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68ca45-2ff0-4d93-87ff-a0c9a4f2839b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:40.525689 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68ca45-2ff0-4d93-87ff-a0c9a4f2839b None admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:40.526969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68ca45-2ff0-4d93-87ff-a0c9a4f2839b None admin] RBAC: Authorizing `identity:get_project(project_id=f94fce41714c4f3099c1ff9eb62caf49)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:40.533285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0660e5f-6f4b-46fe-89c3-839f1bb6656b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:40.535520 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68ca45-2ff0-4d93-87ff-a0c9a4f2839b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:40.537252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 90/184] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:40 2022] GET /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:40.543390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68ca45-2ff0-4d93-87ff-a0c9a4f2839b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:40.607837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada53c9-ed94-4a4a-a9e2-e007ebbe85c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:40.608481 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada53c9-ed94-4a4a-a9e2-e007ebbe85c5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:40.608760 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada53c9-ed94-4a4a-a9e2-e007ebbe85c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:40.609030 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada53c9-ed94-4a4a-a9e2-e007ebbe85c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:40.609768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada53c9-ed94-4a4a-a9e2-e007ebbe85c5 None admin] RBAC: Adding query filter params (role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b, scope.project.id=f94fce41714c4f3099c1ff9eb62caf49, user.id=3f7355d66d79499ba72d14c2a665630a) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:40.610073 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada53c9-ed94-4a4a-a9e2-e007ebbe85c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:40.611602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada53c9-ed94-4a4a-a9e2-e007ebbe85c5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:40.618042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 95/185] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:11:40 2022] GET /identity/v3/role_assignments?user.id=3f7355d66d79499ba72d14c2a665630a&scope.project.id=f94fce41714c4f3099c1ff9eb62caf49&role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:40.675709 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68ca45-2ff0-4d93-87ff-a0c9a4f2839b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:40.712346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69594a47-cb6b-4b05-9acc-02cad0fe536b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:40.713143 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69594a47-cb6b-4b05-9acc-02cad0fe536b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:40.713517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69594a47-cb6b-4b05-9acc-02cad0fe536b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:40.713885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69594a47-cb6b-4b05-9acc-02cad0fe536b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:40.714719 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69594a47-cb6b-4b05-9acc-02cad0fe536b None admin] RBAC: Adding query filter params (role.id=a3287688078248ecb19a6d9c85839631, scope.project.id=f94fce41714c4f3099c1ff9eb62caf49, user.id=3f7355d66d79499ba72d14c2a665630a) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:40.715153 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69594a47-cb6b-4b05-9acc-02cad0fe536b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:40.716693 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69594a47-cb6b-4b05-9acc-02cad0fe536b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:40.724179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 91/186] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:11:40 2022] GET /identity/v3/role_assignments?user.id=3f7355d66d79499ba72d14c2a665630a&scope.project.id=f94fce41714c4f3099c1ff9eb62caf49&role.id=a3287688078248ecb19a6d9c85839631 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:41.310746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8585eca-3e0e-4e50-b661-d034eb11451b None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:41.311296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8585eca-3e0e-4e50-b661-d034eb11451b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:41.311370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8585eca-3e0e-4e50-b661-d034eb11451b None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:41.312711 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 96/187] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:41.318387 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d38b4c-02cb-41e1-8e07-b95827111988 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:41.318825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d38b4c-02cb-41e1-8e07-b95827111988 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:41.318966 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d38b4c-02cb-41e1-8e07-b95827111988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:41.337908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a7d38b4c-02cb-41e1-8e07-b95827111988 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:41.339107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a7d38b4c-02cb-41e1-8e07-b95827111988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:41.398023 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:41.415109 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 92/188] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:41.592154 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcf143-6573-48da-8067-b45074485285 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:41.592438 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcf143-6573-48da-8067-b45074485285 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:41.592621 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcf143-6573-48da-8067-b45074485285 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:41.594109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 97/189] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:11:41.599101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff0b0-d757-4fcc-9610-ee2af115d354 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:41.599390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff0b0-d757-4fcc-9610-ee2af115d354 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:41.599655 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff0b0-d757-4fcc-9610-ee2af115d354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:41.618091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-368ff0b0-d757-4fcc-9610-ee2af115d354 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:41.619279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-368ff0b0-d757-4fcc-9610-ee2af115d354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:41.686070 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 93/190] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:41.694023 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcf143-6573-48da-8067-b45074485285 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:41.803547 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcf143-6573-48da-8067-b45074485285 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:41.806597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:41.839580 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979847b-5988-4a16-9899-e7c09cf474ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:41.840215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979847b-5988-4a16-9899-e7c09cf474ca None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:41.840494 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979847b-5988-4a16-9899-e7c09cf474ca None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:41.840766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979847b-5988-4a16-9899-e7c09cf474ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:41.851102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979847b-5988-4a16-9899-e7c09cf474ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-f979847b-5988-4a16-9899-e7c09cf474ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:41.855898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:41.856738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:41.859860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:41.859860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:41.859860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:41.859860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:41.859860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:41.859860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:41.859860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:41.859860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:41.859860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:41.859860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:41.859860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:11:41.859860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:41.859860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 98/191] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:41.863774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff0b0-d757-4fcc-9610-ee2af115d354 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:41.954780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979847b-5988-4a16-9899-e7c09cf474ca None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:41.973625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff0b0-d757-4fcc-9610-ee2af115d354 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:41.997944 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:42.014942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22dc23-0ba1-4c2a-a31a-9ceb63c4cfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:42.015623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22dc23-0ba1-4c2a-a31a-9ceb63c4cfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:42.015858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22dc23-0ba1-4c2a-a31a-9ceb63c4cfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:42.016128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22dc23-0ba1-4c2a-a31a-9ceb63c4cfb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:42.016794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22dc23-0ba1-4c2a-a31a-9ceb63c4cfb4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:42.017084 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22dc23-0ba1-4c2a-a31a-9ceb63c4cfb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:42.018487 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22dc23-0ba1-4c2a-a31a-9ceb63c4cfb4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:42.031607 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 94/192] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:42.033257 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22dc23-0ba1-4c2a-a31a-9ceb63c4cfb4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:42.061438 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979847b-5988-4a16-9899-e7c09cf474ca None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:42.097580 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c8d2b-588d-4f9d-8e7f-9f80ed36a7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:42.098200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c8d2b-588d-4f9d-8e7f-9f80ed36a7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:42.098471 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c8d2b-588d-4f9d-8e7f-9f80ed36a7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:42.098738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c8d2b-588d-4f9d-8e7f-9f80ed36a7b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:42.099868 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c8d2b-588d-4f9d-8e7f-9f80ed36a7b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:42.101359 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c8d2b-588d-4f9d-8e7f-9f80ed36a7b8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:42.102866 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 99/193] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:42.104001 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff23b5-3004-4992-9f4c-fdb970081d9b None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:42.104277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff23b5-3004-4992-9f4c-fdb970081d9b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:42.104549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff23b5-3004-4992-9f4c-fdb970081d9b None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:42.105885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 100/194] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:42.108301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff23b5-3004-4992-9f4c-fdb970081d9b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:42.136558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22dc23-0ba1-4c2a-a31a-9ceb63c4cfb4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:42.139246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:42.171472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07972c61-27b0-4a98-a62b-132a07c3eb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:42.172100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07972c61-27b0-4a98-a62b-132a07c3eb08 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:42.172378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07972c61-27b0-4a98-a62b-132a07c3eb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:42.172651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07972c61-27b0-4a98-a62b-132a07c3eb08 None admin] PATH_INFO: `/v3/users/c8e730bd8fe549dda89c027ecf895685` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:42.186453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07972c61-27b0-4a98-a62b-132a07c3eb08 None admin] RBAC: Authorizing `identity:get_user(user_id=c8e730bd8fe549dda89c027ecf895685)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:42.187922 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07972c61-27b0-4a98-a62b-132a07c3eb08 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:42.189536 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 95/195] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:42 2022] GET /identity/v3/users/c8e730bd8fe549dda89c027ecf895685 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:42.190827 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6582c76-cba6-4262-8e0e-47bb9dc827e4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:42.191228 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6582c76-cba6-4262-8e0e-47bb9dc827e4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:42.191462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6582c76-cba6-4262-8e0e-47bb9dc827e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:42.208708 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f6582c76-cba6-4262-8e0e-47bb9dc827e4 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:42.209880 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f6582c76-cba6-4262-8e0e-47bb9dc827e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:42.216539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff23b5-3004-4992-9f4c-fdb970081d9b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:42.245173 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:42.251409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226c1fe-adbd-438c-8497-ff90746a2837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:42.252234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226c1fe-adbd-438c-8497-ff90746a2837 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:42.252358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226c1fe-adbd-438c-8497-ff90746a2837 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:42.252567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226c1fe-adbd-438c-8497-ff90746a2837 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:42.253465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226c1fe-adbd-438c-8497-ff90746a2837 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:42.254904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226c1fe-adbd-438c-8497-ff90746a2837 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:42.276797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 101/196] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:11:42 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:11:42.277619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226c1fe-adbd-438c-8497-ff90746a2837 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:42.291124 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 96/197] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:42.382955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:42.399471 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226c1fe-adbd-438c-8497-ff90746a2837 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:42.436802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891c124-9f36-4742-bece-e4fcd72be843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:42.437457 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891c124-9f36-4742-bece-e4fcd72be843 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:42.437737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891c124-9f36-4742-bece-e4fcd72be843 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:42.438012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891c124-9f36-4742-bece-e4fcd72be843 None admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:42.439186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891c124-9f36-4742-bece-e4fcd72be843 None admin] RBAC: Authorizing `identity:get_project(project_id=f94fce41714c4f3099c1ff9eb62caf49)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:42.441479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891c124-9f36-4742-bece-e4fcd72be843 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:42.443233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 102/198] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:42 2022] GET /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:42.448852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6582c76-cba6-4262-8e0e-47bb9dc827e4 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:42.546732 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891c124-9f36-4742-bece-e4fcd72be843 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:42.559445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6582c76-cba6-4262-8e0e-47bb9dc827e4 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:42.607442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121abd0-3bfa-4fb2-b19b-fbfcf74485e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:42.608117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121abd0-3bfa-4fb2-b19b-fbfcf74485e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:42.608396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121abd0-3bfa-4fb2-b19b-fbfcf74485e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:42.608670 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121abd0-3bfa-4fb2-b19b-fbfcf74485e5 None admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49/users/c8e730bd8fe549dda89c027ecf895685/roles/062b4d142bbc4085acd6efc13e6b2b74` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:42.610643 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121abd0-3bfa-4fb2-b19b-fbfcf74485e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f94fce41714c4f3099c1ff9eb62caf49, user_id=c8e730bd8fe549dda89c027ecf895685, role_id=062b4d142bbc4085acd6efc13e6b2b74)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:42.612668 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121abd0-3bfa-4fb2-b19b-fbfcf74485e5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:42.629902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 97/199] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:11:42 2022] PUT /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49/users/c8e730bd8fe549dda89c027ecf895685/roles/062b4d142bbc4085acd6efc13e6b2b74 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:11:42.671930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891c124-9f36-4742-bece-e4fcd72be843 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:42.711168 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf5ade-ee92-49d4-ab31-8d80708ff618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:42.711835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf5ade-ee92-49d4-ab31-8d80708ff618 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:42.712119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf5ade-ee92-49d4-ab31-8d80708ff618 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:42.712397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf5ade-ee92-49d4-ab31-8d80708ff618 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:42.728766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf5ade-ee92-49d4-ab31-8d80708ff618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-51cf5ade-ee92-49d4-ab31-8d80708ff618 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:42.734497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:42.735449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:42.736216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:42.736216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:42.736216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:42.736216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:42.736216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:42.736216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:42.736216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:42.736216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:42.736216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:42.736216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:42.736216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:42.736216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:42.741694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 103/200] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:42.747841 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121abd0-3bfa-4fb2-b19b-fbfcf74485e5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:42.785525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:42.861756 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121abd0-3bfa-4fb2-b19b-fbfcf74485e5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:42.896632 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90d43f-edcd-4778-9309-b7002a840dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:42.897246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90d43f-edcd-4778-9309-b7002a840dfc None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:42.897523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90d43f-edcd-4778-9309-b7002a840dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:42.897803 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90d43f-edcd-4778-9309-b7002a840dfc None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:42.898464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90d43f-edcd-4778-9309-b7002a840dfc None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:42.898822 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90d43f-edcd-4778-9309-b7002a840dfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:42.900294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90d43f-edcd-4778-9309-b7002a840dfc None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:42.913012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 98/201] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:42.919193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf5ade-ee92-49d4-ab31-8d80708ff618 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:42.938617 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:43.023476 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf5ade-ee92-49d4-ab31-8d80708ff618 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:43.069106 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28937e59-416f-4a43-b9f4-eab0234b6e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:43.070113 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28937e59-416f-4a43-b9f4-eab0234b6e9c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:43.070585 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28937e59-416f-4a43-b9f4-eab0234b6e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:43.071058 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28937e59-416f-4a43-b9f4-eab0234b6e9c None admin] PATH_INFO: `/v3/users/1d28c2dd35f44cd89b1159a51aba33f8` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:43.072433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28937e59-416f-4a43-b9f4-eab0234b6e9c None admin] RBAC: Authorizing `identity:get_user(user_id=1d28c2dd35f44cd89b1159a51aba33f8)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:43.074071 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28937e59-416f-4a43-b9f4-eab0234b6e9c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:43.076191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 104/202] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:42 2022] GET /identity/v3/users/1d28c2dd35f44cd89b1159a51aba33f8 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:43.082409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90d43f-edcd-4778-9309-b7002a840dfc None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:43.177562 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:43.187221 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90d43f-edcd-4778-9309-b7002a840dfc None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:43.223578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6a0a2-0f87-41e5-afbf-16787bcfec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:43.224113 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6a0a2-0f87-41e5-afbf-16787bcfec96 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:43.224393 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6a0a2-0f87-41e5-afbf-16787bcfec96 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:43.224673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6a0a2-0f87-41e5-afbf-16787bcfec96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:43.296231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6a0a2-0f87-41e5-afbf-16787bcfec96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:43.297715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6a0a2-0f87-41e5-afbf-16787bcfec96 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:43.394794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:43.407533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 99/203] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:11:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:11:43.414310 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28937e59-416f-4a43-b9f4-eab0234b6e9c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:43.505248 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:43.521283 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28937e59-416f-4a43-b9f4-eab0234b6e9c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:43.555285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b584d-7d0b-40d9-85ef-28fa771d6a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:43.555921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b584d-7d0b-40d9-85ef-28fa771d6a01 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:43.556198 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b584d-7d0b-40d9-85ef-28fa771d6a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:43.556470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b584d-7d0b-40d9-85ef-28fa771d6a01 None admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:43.557584 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b584d-7d0b-40d9-85ef-28fa771d6a01 None admin] RBAC: Authorizing `identity:get_project(project_id=f94fce41714c4f3099c1ff9eb62caf49)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:43.559030 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b584d-7d0b-40d9-85ef-28fa771d6a01 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:43.560891 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 105/204] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:43 2022] GET /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:43.567255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6a0a2-0f87-41e5-afbf-16787bcfec96 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:43.670750 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6a0a2-0f87-41e5-afbf-16787bcfec96 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:43.705862 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed2673-db3e-47e6-af5f-7b13db4b6c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:43.706630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed2673-db3e-47e6-af5f-7b13db4b6c9b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:43.706908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed2673-db3e-47e6-af5f-7b13db4b6c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:43.707201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed2673-db3e-47e6-af5f-7b13db4b6c9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:43.707932 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed2673-db3e-47e6-af5f-7b13db4b6c9b None admin] RBAC: Adding query filter params (role.id=e460c7cf7b5b4c35a5a0e326f69a9fe3, scope.project.id=f94fce41714c4f3099c1ff9eb62caf49, user.id=1d28c2dd35f44cd89b1159a51aba33f8) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:43.708235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed2673-db3e-47e6-af5f-7b13db4b6c9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:43.709660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed2673-db3e-47e6-af5f-7b13db4b6c9b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:43.715863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 100/205] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:11:43 2022] GET /identity/v3/role_assignments?user.id=1d28c2dd35f44cd89b1159a51aba33f8&scope.project.id=f94fce41714c4f3099c1ff9eb62caf49&role.id=e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:44.644325 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81362eb1-2300-44e6-bfde-ac506bcabfdb None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:44.644856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81362eb1-2300-44e6-bfde-ac506bcabfdb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:44.644856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81362eb1-2300-44e6-bfde-ac506bcabfdb None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:44.646586 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 106/206] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:44.652397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575038a-b23d-4535-b219-6b369d1c531f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:44.652874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575038a-b23d-4535-b219-6b369d1c531f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:44.652942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575038a-b23d-4535-b219-6b369d1c531f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:44.671109 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4575038a-b23d-4535-b219-6b369d1c531f None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:44.672314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4575038a-b23d-4535-b219-6b369d1c531f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:44.730040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:44.745077 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb122-fd22-4624-983e-682532f374d3 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:44.745729 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb122-fd22-4624-983e-682532f374d3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:44.745729 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb122-fd22-4624-983e-682532f374d3 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:44.747199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 107/207] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:44.750678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 101/208] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:44.753664 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00088aec-795d-43fc-bd2a-35b97af9d15e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:44.753948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00088aec-795d-43fc-bd2a-35b97af9d15e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:44.754220 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00088aec-795d-43fc-bd2a-35b97af9d15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:44.773149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-00088aec-795d-43fc-bd2a-35b97af9d15e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:44.774377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-00088aec-795d-43fc-bd2a-35b97af9d15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:44.842313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 108/209] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:44.861164 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9384ed4-d720-4281-a71b-a0a738a7105e None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:44.861551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9384ed4-d720-4281-a71b-a0a738a7105e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:44.862388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9384ed4-d720-4281-a71b-a0a738a7105e None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:44.864324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 102/210] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:44.869387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69f007-8288-43c3-9749-fc288ee6326e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:44.869672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69f007-8288-43c3-9749-fc288ee6326e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:44.870004 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69f007-8288-43c3-9749-fc288ee6326e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:44.888178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-dd69f007-8288-43c3-9749-fc288ee6326e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:44.889343 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dd69f007-8288-43c3-9749-fc288ee6326e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:44.955248 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 109/211] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:11:45.025809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9384ed4-d720-4281-a71b-a0a738a7105e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:45.110802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69f007-8288-43c3-9749-fc288ee6326e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:45.125845 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:45.132059 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9384ed4-d720-4281-a71b-a0a738a7105e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:45.166428 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe11b8d-b264-4e5c-a147-e98788279229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:45.167103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe11b8d-b264-4e5c-a147-e98788279229 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:45.167364 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe11b8d-b264-4e5c-a147-e98788279229 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:45.167634 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe11b8d-b264-4e5c-a147-e98788279229 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:45.177962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe11b8d-b264-4e5c-a147-e98788279229 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-cbe11b8d-b264-4e5c-a147-e98788279229 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:45.182729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:45.183735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:45.184502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:45.184502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:45.184502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:45.184502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:45.184502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:45.184502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:45.184502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:45.184502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:45.184502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:45.184502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:45.184502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:45.184502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:45.184502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 103/212] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:45.189507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe11b8d-b264-4e5c-a147-e98788279229 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:45.225783 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69f007-8288-43c3-9749-fc288ee6326e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:45.240321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:45.260231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1c765-e4f7-4676-a14e-af198ad9d478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:45.260864 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1c765-e4f7-4676-a14e-af198ad9d478 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:45.261140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1c765-e4f7-4676-a14e-af198ad9d478 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:45.261404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1c765-e4f7-4676-a14e-af198ad9d478 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:45.262517 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1c765-e4f7-4676-a14e-af198ad9d478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:45.263979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1c765-e4f7-4676-a14e-af198ad9d478 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:45.265536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 110/213] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:45.267222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1c765-e4f7-4676-a14e-af198ad9d478 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:45.294766 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe11b8d-b264-4e5c-a147-e98788279229 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:45.327437 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecabfeb-3044-47e4-b1d3-81bad17f4b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:45.328039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecabfeb-3044-47e4-b1d3-81bad17f4b7d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:45.328313 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecabfeb-3044-47e4-b1d3-81bad17f4b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:45.328582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecabfeb-3044-47e4-b1d3-81bad17f4b7d None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:45.329239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecabfeb-3044-47e4-b1d3-81bad17f4b7d None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:45.329535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecabfeb-3044-47e4-b1d3-81bad17f4b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:45.331555 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecabfeb-3044-47e4-b1d3-81bad17f4b7d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:45.339386 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 104/214] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:45.340150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecabfeb-3044-47e4-b1d3-81bad17f4b7d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:45.346457 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:45.366282 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1c765-e4f7-4676-a14e-af198ad9d478 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:45.400420 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7fd7e-967e-4efc-be3e-73c123c91331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:45.401056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7fd7e-967e-4efc-be3e-73c123c91331 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:45.401338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7fd7e-967e-4efc-be3e-73c123c91331 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:45.401614 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7fd7e-967e-4efc-be3e-73c123c91331 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:45.412348 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7fd7e-967e-4efc-be3e-73c123c91331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-a6c7fd7e-967e-4efc-be3e-73c123c91331 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:45.417158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:45.418002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:45.419659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:45.419659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:45.419659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:45.419659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:45.419659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:45.419659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:45.419659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:45.419659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:45.419659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:45.419659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:45.419659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:11:45.419659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:45.419659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 111/215] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:45.419659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7fd7e-967e-4efc-be3e-73c123c91331 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:45.440503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecabfeb-3044-47e4-b1d3-81bad17f4b7d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:45.448965 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:45.474661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f790276-c132-4b18-9afd-2302843c4e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:45.475299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f790276-c132-4b18-9afd-2302843c4e23 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:45.475584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f790276-c132-4b18-9afd-2302843c4e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:45.475902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f790276-c132-4b18-9afd-2302843c4e23 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:45.476509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f790276-c132-4b18-9afd-2302843c4e23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:45.477947 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f790276-c132-4b18-9afd-2302843c4e23 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:45.498107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 105/216] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:11:45 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:45.504233 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f790276-c132-4b18-9afd-2302843c4e23 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:45.522625 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7fd7e-967e-4efc-be3e-73c123c91331 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:45.564014 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920a80e-88a1-4e97-a33c-8efa3eead6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:45.565791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920a80e-88a1-4e97-a33c-8efa3eead6be None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:45.566347 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920a80e-88a1-4e97-a33c-8efa3eead6be None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:45.566888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920a80e-88a1-4e97-a33c-8efa3eead6be None admin] PATH_INFO: `/v3/users/0dbaa80f37d34d028ab3e6f0a4fd25ec` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:45.579418 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:45.583094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920a80e-88a1-4e97-a33c-8efa3eead6be None admin] RBAC: Authorizing `identity:get_user(user_id=0dbaa80f37d34d028ab3e6f0a4fd25ec)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:45.586232 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920a80e-88a1-4e97-a33c-8efa3eead6be None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:45.588200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 112/217] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:45 2022] GET /identity/v3/users/0dbaa80f37d34d028ab3e6f0a4fd25ec => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:45.594272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920a80e-88a1-4e97-a33c-8efa3eead6be None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:45.623926 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f790276-c132-4b18-9afd-2302843c4e23 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:45.661338 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953afac-840f-4c10-84bf-501feb518cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:45.662651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953afac-840f-4c10-84bf-501feb518cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:45.662979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953afac-840f-4c10-84bf-501feb518cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:45.663322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953afac-840f-4c10-84bf-501feb518cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:45.664003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953afac-840f-4c10-84bf-501feb518cd4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:45.664301 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953afac-840f-4c10-84bf-501feb518cd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:45.665692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953afac-840f-4c10-84bf-501feb518cd4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:45.676291 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 106/218] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:45.681618 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953afac-840f-4c10-84bf-501feb518cd4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:45.713270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:45.718404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920a80e-88a1-4e97-a33c-8efa3eead6be None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:45.759478 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684f7fe-3486-4b16-b246-1d41eac7ae85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:45.760101 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684f7fe-3486-4b16-b246-1d41eac7ae85 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:45.760380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684f7fe-3486-4b16-b246-1d41eac7ae85 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:45.760651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684f7fe-3486-4b16-b246-1d41eac7ae85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:45.761391 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684f7fe-3486-4b16-b246-1d41eac7ae85 None admin] RBAC: Adding query filter params (role.id=e460c7cf7b5b4c35a5a0e326f69a9fe3, user.id=0dbaa80f37d34d028ab3e6f0a4fd25ec, scope.system=all) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:45.761715 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684f7fe-3486-4b16-b246-1d41eac7ae85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:45.763143 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684f7fe-3486-4b16-b246-1d41eac7ae85 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:45.771341 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 113/219] 10.222.0.18 () {56 vars in 1308 bytes} [Fri Oct 14 02:11:45 2022] GET /identity/v3/role_assignments?user.id=0dbaa80f37d34d028ab3e6f0a4fd25ec&scope.system=all&role.id=e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:45.814853 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953afac-840f-4c10-84bf-501feb518cd4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:45.856529 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10face1a-fed8-4e72-8cd6-a7d4b64f98b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:45.857179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10face1a-fed8-4e72-8cd6-a7d4b64f98b4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:45.857457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10face1a-fed8-4e72-8cd6-a7d4b64f98b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:45.857727 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10face1a-fed8-4e72-8cd6-a7d4b64f98b4 None admin] PATH_INFO: `/v3/users/c8e730bd8fe549dda89c027ecf895685` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:45.858915 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10face1a-fed8-4e72-8cd6-a7d4b64f98b4 None admin] RBAC: Authorizing `identity:get_user(user_id=c8e730bd8fe549dda89c027ecf895685)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:45.860454 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10face1a-fed8-4e72-8cd6-a7d4b64f98b4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:45.862543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 107/220] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:45 2022] GET /identity/v3/users/c8e730bd8fe549dda89c027ecf895685 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:45.868219 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684f7fe-3486-4b16-b246-1d41eac7ae85 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:45.941267 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:45.985371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684f7fe-3486-4b16-b246-1d41eac7ae85 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:46.021925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b30d5-2551-42b5-8e85-f2e16373b26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:46.022562 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b30d5-2551-42b5-8e85-f2e16373b26d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:46.022847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b30d5-2551-42b5-8e85-f2e16373b26d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:46.023135 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b30d5-2551-42b5-8e85-f2e16373b26d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:46.093485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b30d5-2551-42b5-8e85-f2e16373b26d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:46.095001 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b30d5-2551-42b5-8e85-f2e16373b26d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:46.157454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:46.198892 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 114/221] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:11:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 14 02:11:46.204210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10face1a-fed8-4e72-8cd6-a7d4b64f98b4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:46.313909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:46.321712 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10face1a-fed8-4e72-8cd6-a7d4b64f98b4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:46.357019 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb608a21-bdb7-4dd3-9002-fc9ce61e9085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:46.357648 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb608a21-bdb7-4dd3-9002-fc9ce61e9085 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:46.357922 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb608a21-bdb7-4dd3-9002-fc9ce61e9085 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:46.358193 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb608a21-bdb7-4dd3-9002-fc9ce61e9085 None admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:46.359352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb608a21-bdb7-4dd3-9002-fc9ce61e9085 None admin] RBAC: Authorizing `identity:get_project(project_id=f94fce41714c4f3099c1ff9eb62caf49)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:46.360868 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb608a21-bdb7-4dd3-9002-fc9ce61e9085 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:46.363577 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 108/222] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:11:46 2022] GET /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:46.369142 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b30d5-2551-42b5-8e85-f2e16373b26d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:46.479107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b30d5-2551-42b5-8e85-f2e16373b26d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:46.514519 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fffd68b-0aa3-49a2-80fa-08f4f4b18be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:46.515169 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffd68b-0aa3-49a2-80fa-08f4f4b18be8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:46.515454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffd68b-0aa3-49a2-80fa-08f4f4b18be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:46.515734 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffd68b-0aa3-49a2-80fa-08f4f4b18be8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:46.516469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fffd68b-0aa3-49a2-80fa-08f4f4b18be8 None admin] RBAC: Adding query filter params (role.id=062b4d142bbc4085acd6efc13e6b2b74, scope.project.id=f94fce41714c4f3099c1ff9eb62caf49, user.id=c8e730bd8fe549dda89c027ecf895685) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:46.516782 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fffd68b-0aa3-49a2-80fa-08f4f4b18be8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:46.518399 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fffd68b-0aa3-49a2-80fa-08f4f4b18be8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:46.524967 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 115/223] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:11:46 2022] GET /identity/v3/role_assignments?user.id=c8e730bd8fe549dda89c027ecf895685&scope.project.id=f94fce41714c4f3099c1ff9eb62caf49&role.id=062b4d142bbc4085acd6efc13e6b2b74 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:47.843485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee847e-57e0-4258-9d3d-f2ba91f6d5ac None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:47.844065 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee847e-57e0-4258-9d3d-f2ba91f6d5ac None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:47.844065 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee847e-57e0-4258-9d3d-f2ba91f6d5ac None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:47.845487 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 109/224] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:47.851873 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a0df5-7762-4b22-a3e5-97a83bd37ff6 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:47.852835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a0df5-7762-4b22-a3e5-97a83bd37ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:47.853128 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a0df5-7762-4b22-a3e5-97a83bd37ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:47.873133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-cb8a0df5-7762-4b22-a3e5-97a83bd37ff6 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:47.874617 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cb8a0df5-7762-4b22-a3e5-97a83bd37ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:47.897818 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:47.947133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 116/225] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:47.947314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a316ab-8a4d-4013-b4e7-b2cfc32d3e47 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:47.947566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a316ab-8a4d-4013-b4e7-b2cfc32d3e47 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:47.948005 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a316ab-8a4d-4013-b4e7-b2cfc32d3e47 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:47.949344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 110/226] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:47.954580 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58393599-6610-472f-9c62-5f7df49c4d50 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:47.954869 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58393599-6610-472f-9c62-5f7df49c4d50 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:47.955193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58393599-6610-472f-9c62-5f7df49c4d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:47.968450 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1716f9-49e8-41a3-99cc-10bbfe8300ca None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:47.968727 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1716f9-49e8-41a3-99cc-10bbfe8300ca None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:47.969002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1716f9-49e8-41a3-99cc-10bbfe8300ca None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:47.970346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 111/227] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:11:47.974179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-58393599-6610-472f-9c62-5f7df49c4d50 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:47.975404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-58393599-6610-472f-9c62-5f7df49c4d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:47.975838 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da1aa9-88f8-446a-8974-a5ec933915f7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:47.976126 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da1aa9-88f8-446a-8974-a5ec933915f7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:47.976401 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da1aa9-88f8-446a-8974-a5ec933915f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:47.996116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-45da1aa9-88f8-446a-8974-a5ec933915f7 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:47.997213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-45da1aa9-88f8-446a-8974-a5ec933915f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:48.043989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 117/228] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:48.065695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 112/229] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:11:48.216507 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58393599-6610-472f-9c62-5f7df49c4d50 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:48.309109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:48.321257 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da1aa9-88f8-446a-8974-a5ec933915f7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:48.325157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58393599-6610-472f-9c62-5f7df49c4d50 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:48.360576 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425601c-7c58-461d-99f6-fb0569c12e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:48.361214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425601c-7c58-461d-99f6-fb0569c12e21 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:48.361491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425601c-7c58-461d-99f6-fb0569c12e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:48.361797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425601c-7c58-461d-99f6-fb0569c12e21 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:48.372088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425601c-7c58-461d-99f6-fb0569c12e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-a425601c-7c58-461d-99f6-fb0569c12e21 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:48.377144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:48.378131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:48.383387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:48.383387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:48.383387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:48.383387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:48.383387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:48.383387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:48.383387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:48.383387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:48.383387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:48.383387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:48.383387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:11:48.383387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:48.383387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 118/230] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:48.383387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425601c-7c58-461d-99f6-fb0569c12e21 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:48.414937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:48.423930 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da1aa9-88f8-446a-8974-a5ec933915f7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:48.458398 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb192ff-0127-4cd5-bc48-5083f1bb5263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:48.459020 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb192ff-0127-4cd5-bc48-5083f1bb5263 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:48.459324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb192ff-0127-4cd5-bc48-5083f1bb5263 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:48.459591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb192ff-0127-4cd5-bc48-5083f1bb5263 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:48.460672 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb192ff-0127-4cd5-bc48-5083f1bb5263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:48.462114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb192ff-0127-4cd5-bc48-5083f1bb5263 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:48.463780 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 113/231] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:48.464532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb192ff-0127-4cd5-bc48-5083f1bb5263 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:48.482847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425601c-7c58-461d-99f6-fb0569c12e21 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:48.516496 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d0a29-d3d1-4ee3-bd81-1930ca47c39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:48.517125 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d0a29-d3d1-4ee3-bd81-1930ca47c39e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:48.517411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d0a29-d3d1-4ee3-bd81-1930ca47c39e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:48.517681 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d0a29-d3d1-4ee3-bd81-1930ca47c39e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:48.528666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d0a29-d3d1-4ee3-bd81-1930ca47c39e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-a85d0a29-d3d1-4ee3-bd81-1930ca47c39e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:48.533417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:48.534969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:48.535811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:48.535811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:48.535811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:48.535811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:48.535811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:48.535811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:48.535811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:48.535811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:48.535811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:48.535811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:48.535811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:48.535811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:48.535811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 119/232] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:48.536472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d0a29-d3d1-4ee3-bd81-1930ca47c39e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:48.547638 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:48.566527 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb192ff-0127-4cd5-bc48-5083f1bb5263 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:48.600310 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f95a53-66d7-42fa-94c2-8a82df232171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:48.600918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f95a53-66d7-42fa-94c2-8a82df232171 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:48.601198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f95a53-66d7-42fa-94c2-8a82df232171 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:48.601470 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f95a53-66d7-42fa-94c2-8a82df232171 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:48.602133 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f95a53-66d7-42fa-94c2-8a82df232171 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:48.602437 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f95a53-66d7-42fa-94c2-8a82df232171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:48.603843 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f95a53-66d7-42fa-94c2-8a82df232171 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:48.612460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 114/233] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:48.612592 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f95a53-66d7-42fa-94c2-8a82df232171 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:48.639511 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d0a29-d3d1-4ee3-bd81-1930ca47c39e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:48.639794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:48.673558 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba2214-6411-43e3-a7c9-1314285c2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:48.674264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba2214-6411-43e3-a7c9-1314285c2fbe None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:48.674991 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba2214-6411-43e3-a7c9-1314285c2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:48.675299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba2214-6411-43e3-a7c9-1314285c2fbe None admin] PATH_INFO: `/v3/groups` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:48.675930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba2214-6411-43e3-a7c9-1314285c2fbe None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:48.677401 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba2214-6411-43e3-a7c9-1314285c2fbe None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:48.693408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 120/234] 10.222.0.18 () {60 vars in 1134 bytes} [Fri Oct 14 02:11:48 2022] POST /identity/v3/groups => generated 229 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:48.694230 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba2214-6411-43e3-a7c9-1314285c2fbe None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:48.719369 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f95a53-66d7-42fa-94c2-8a82df232171 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:48.760988 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ded85-31f2-48c3-8839-5a1b5a91d896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:48.761715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ded85-31f2-48c3-8839-5a1b5a91d896 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:48.762058 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ded85-31f2-48c3-8839-5a1b5a91d896 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:48.762403 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ded85-31f2-48c3-8839-5a1b5a91d896 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:48.763221 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ded85-31f2-48c3-8839-5a1b5a91d896 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:48.763529 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ded85-31f2-48c3-8839-5a1b5a91d896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:48.764923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ded85-31f2-48c3-8839-5a1b5a91d896 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:48.774302 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 115/235] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:48.779947 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ded85-31f2-48c3-8839-5a1b5a91d896 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:48.781496 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:48.825814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba2214-6411-43e3-a7c9-1314285c2fbe None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:48.868327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d672d-36c4-4401-b41f-74c325472d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:48.868971 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d672d-36c4-4401-b41f-74c325472d16 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:48.869239 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d672d-36c4-4401-b41f-74c325472d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:48.869508 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d672d-36c4-4401-b41f-74c325472d16 None admin] PATH_INFO: `/v3/users/7b70d725813d4e5aa8690d4d7d37d9ff` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:48.881587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:48.890443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d672d-36c4-4401-b41f-74c325472d16 None admin] RBAC: Authorizing `identity:get_user(user_id=7b70d725813d4e5aa8690d4d7d37d9ff)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:48.892314 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d672d-36c4-4401-b41f-74c325472d16 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:48.894605 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 121/236] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:48 2022] GET /identity/v3/users/7b70d725813d4e5aa8690d4d7d37d9ff => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:48.900458 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d672d-36c4-4401-b41f-74c325472d16 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:48.901266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ded85-31f2-48c3-8839-5a1b5a91d896 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:48.940906 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a63d55-2e64-4c47-8748-db5727b3d3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:48.941541 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a63d55-2e64-4c47-8748-db5727b3d3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:48.941818 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a63d55-2e64-4c47-8748-db5727b3d3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:48.942142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a63d55-2e64-4c47-8748-db5727b3d3e3 None admin] PATH_INFO: `/v3/users/0dbaa80f37d34d028ab3e6f0a4fd25ec` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:48.943328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a63d55-2e64-4c47-8748-db5727b3d3e3 None admin] RBAC: Authorizing `identity:get_user(user_id=0dbaa80f37d34d028ab3e6f0a4fd25ec)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:48.945132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a63d55-2e64-4c47-8748-db5727b3d3e3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:48.946578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 116/237] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:48 2022] GET /identity/v3/users/0dbaa80f37d34d028ab3e6f0a4fd25ec => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:48.951871 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a63d55-2e64-4c47-8748-db5727b3d3e3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:49.011427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d672d-36c4-4401-b41f-74c325472d16 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:49.035434 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:49.049806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eba8d8-1942-4c35-8d5e-ee1f00b81863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:49.050882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eba8d8-1942-4c35-8d5e-ee1f00b81863 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:49.051199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eba8d8-1942-4c35-8d5e-ee1f00b81863 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:49.051477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eba8d8-1942-4c35-8d5e-ee1f00b81863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:49.052210 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eba8d8-1942-4c35-8d5e-ee1f00b81863 None admin] RBAC: Adding query filter params (role.id=062b4d142bbc4085acd6efc13e6b2b74, user.id=7b70d725813d4e5aa8690d4d7d37d9ff, scope.system=all) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:49.052518 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eba8d8-1942-4c35-8d5e-ee1f00b81863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:49.053949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eba8d8-1942-4c35-8d5e-ee1f00b81863 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:49.059123 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 122/238] 10.222.0.18 () {56 vars in 1308 bytes} [Fri Oct 14 02:11:48 2022] GET /identity/v3/role_assignments?user.id=7b70d725813d4e5aa8690d4d7d37d9ff&scope.system=all&role.id=062b4d142bbc4085acd6efc13e6b2b74 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:49.060750 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a63d55-2e64-4c47-8748-db5727b3d3e3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:49.099774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2adaa5b-fd7a-45bd-954b-c6d9502799ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:49.100371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adaa5b-fd7a-45bd-954b-c6d9502799ef None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:49.100650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adaa5b-fd7a-45bd-954b-c6d9502799ef None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:49.100925 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adaa5b-fd7a-45bd-954b-c6d9502799ef None admin] PATH_INFO: `/v3/system/users/0dbaa80f37d34d028ab3e6f0a4fd25ec/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:49.102483 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2adaa5b-fd7a-45bd-954b-c6d9502799ef None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0dbaa80f37d34d028ab3e6f0a4fd25ec, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:49.103919 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2adaa5b-fd7a-45bd-954b-c6d9502799ef None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:49.115390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 117/239] 10.222.0.18 () {58 vars in 1334 bytes} [Fri Oct 14 02:11:48 2022] PUT /identity/v3/system/users/0dbaa80f37d34d028ab3e6f0a4fd25ec/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:11:50.902214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e10ca-e932-4bdd-8e3b-87bc39032ddd None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:50.902716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e10ca-e932-4bdd-8e3b-87bc39032ddd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:50.902821 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e10ca-e932-4bdd-8e3b-87bc39032ddd None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:50.904293 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 123/240] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:50.910009 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a953bbc-9e3b-4005-8b21-897b5970f13c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:50.910434 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a953bbc-9e3b-4005-8b21-897b5970f13c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:50.910630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a953bbc-9e3b-4005-8b21-897b5970f13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:50.929026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8a953bbc-9e3b-4005-8b21-897b5970f13c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:50.930214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8a953bbc-9e3b-4005-8b21-897b5970f13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:50.941624 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:50.999795 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 118/241] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:51.193910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b637345-b886-4b2d-a896-815e2e4b2791 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:51.194261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b637345-b886-4b2d-a896-815e2e4b2791 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:51.194455 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b637345-b886-4b2d-a896-815e2e4b2791 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:51.195951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 124/242] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:51.200777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1e566-2b14-4dc9-9bb5-e517e717e9e3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:51.201063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1e566-2b14-4dc9-9bb5-e517e717e9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:51.201337 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1e566-2b14-4dc9-9bb5-e517e717e9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:51.219988 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-09e1e566-2b14-4dc9-9bb5-e517e717e9e3 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:51.221716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-09e1e566-2b14-4dc9-9bb5-e517e717e9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:51.266202 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b637345-b886-4b2d-a896-815e2e4b2791 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:51.288529 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 119/243] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:51.290051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd5bfd-3149-48b6-951d-1a11cee8ffac None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:51.290335 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd5bfd-3149-48b6-951d-1a11cee8ffac None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:51.290605 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd5bfd-3149-48b6-951d-1a11cee8ffac None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:51.291888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 120/244] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:51.296959 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8142565-7ac4-48a0-bd2b-a1a61d3afe50 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:51.297244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8142565-7ac4-48a0-bd2b-a1a61d3afe50 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:51.297519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8142565-7ac4-48a0-bd2b-a1a61d3afe50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:51.315918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f8142565-7ac4-48a0-bd2b-a1a61d3afe50 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:51.317102 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f8142565-7ac4-48a0-bd2b-a1a61d3afe50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:51.384597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 121/245] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:11:51.392731 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b637345-b886-4b2d-a896-815e2e4b2791 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:51.430486 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5e6fd-8c2f-45cf-8142-566bfe28b84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:51.431136 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5e6fd-8c2f-45cf-8142-566bfe28b84e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:51.431413 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5e6fd-8c2f-45cf-8142-566bfe28b84e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:51.431679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5e6fd-8c2f-45cf-8142-566bfe28b84e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:51.432759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5e6fd-8c2f-45cf-8142-566bfe28b84e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:51.434199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5e6fd-8c2f-45cf-8142-566bfe28b84e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:51.436087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 125/246] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:11:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:51.441418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8142565-7ac4-48a0-bd2b-a1a61d3afe50 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:51.450532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:51.541950 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8142565-7ac4-48a0-bd2b-a1a61d3afe50 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:51.551950 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5e6fd-8c2f-45cf-8142-566bfe28b84e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:51.577029 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894b3f5-4ca0-4a9c-b7e4-8b3f8edd8778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:51.577638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894b3f5-4ca0-4a9c-b7e4-8b3f8edd8778 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:51.577919 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894b3f5-4ca0-4a9c-b7e4-8b3f8edd8778 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:51.578184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894b3f5-4ca0-4a9c-b7e4-8b3f8edd8778 None admin] PATH_INFO: `/v3/groups` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:51.578826 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894b3f5-4ca0-4a9c-b7e4-8b3f8edd8778 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:51.580359 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894b3f5-4ca0-4a9c-b7e4-8b3f8edd8778 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:51.597163 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 122/247] 10.222.0.18 () {60 vars in 1134 bytes} [Fri Oct 14 02:11:51 2022] POST /identity/v3/groups => generated 226 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:11:51.612579 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:51.655920 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894b3f5-4ca0-4a9c-b7e4-8b3f8edd8778 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:51.664178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5e6fd-8c2f-45cf-8142-566bfe28b84e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:51.701489 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2522d2-4401-453e-b82c-d63df0e5e7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:51.702140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2522d2-4401-453e-b82c-d63df0e5e7ac None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:51.702409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2522d2-4401-453e-b82c-d63df0e5e7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:51.702680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2522d2-4401-453e-b82c-d63df0e5e7ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:51.714902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2522d2-4401-453e-b82c-d63df0e5e7ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-dd2522d2-4401-453e-b82c-d63df0e5e7ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:51.719856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:51.720775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:51.721814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:51.721814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:51.721814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:51.721814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:51.721814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:51.721814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:51.721814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:51.721814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:51.721814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:51.721814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:51.721814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:11:51.721814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:51.721814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 126/248] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:11:51.727552 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2522d2-4401-453e-b82c-d63df0e5e7ac None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:51.735095 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:51.777513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894b3f5-4ca0-4a9c-b7e4-8b3f8edd8778 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:51.819978 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967e22d-8801-4297-945b-60886a068a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:51.820564 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967e22d-8801-4297-945b-60886a068a6e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:51.820833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967e22d-8801-4297-945b-60886a068a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:51.821080 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967e22d-8801-4297-945b-60886a068a6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:51.833765 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967e22d-8801-4297-945b-60886a068a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-5967e22d-8801-4297-945b-60886a068a6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:51.838743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:51.839651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:51.840424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:51.840424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:51.840424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:51.840424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:51.840424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:51.840424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:51.840424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:51.840424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:51.840424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:51.840424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:51.840424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:51.840424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:51.840424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 123/249] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:11:51.847277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2522d2-4401-453e-b82c-d63df0e5e7ac None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:51.847506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967e22d-8801-4297-945b-60886a068a6e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:51.865350 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:51.886868 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539ce87-2c81-490d-b2e0-f33482d1c16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:51.887521 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539ce87-2c81-490d-b2e0-f33482d1c16f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:51.887825 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539ce87-2c81-490d-b2e0-f33482d1c16f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:51.888096 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539ce87-2c81-490d-b2e0-f33482d1c16f None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:51.888765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539ce87-2c81-490d-b2e0-f33482d1c16f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:51.889069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539ce87-2c81-490d-b2e0-f33482d1c16f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:51.890447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539ce87-2c81-490d-b2e0-f33482d1c16f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:51.901029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 127/250] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:51.906613 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539ce87-2c81-490d-b2e0-f33482d1c16f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:51.966562 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967e22d-8801-4297-945b-60886a068a6e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:51.998627 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:52.013786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d7126-f208-4d83-8cff-3f0d2cea2e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:52.014555 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d7126-f208-4d83-8cff-3f0d2cea2e10 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:52.014934 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d7126-f208-4d83-8cff-3f0d2cea2e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:52.016377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d7126-f208-4d83-8cff-3f0d2cea2e10 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:52.016670 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d7126-f208-4d83-8cff-3f0d2cea2e10 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:52.016979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d7126-f208-4d83-8cff-3f0d2cea2e10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:52.019524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d7126-f208-4d83-8cff-3f0d2cea2e10 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:52.032281 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539ce87-2c81-490d-b2e0-f33482d1c16f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:52.035471 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 124/251] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:52.036426 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d7126-f208-4d83-8cff-3f0d2cea2e10 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:52.076089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70888f0-839b-48bb-a2f5-fa1454b035e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:52.076672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70888f0-839b-48bb-a2f5-fa1454b035e3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:52.076954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70888f0-839b-48bb-a2f5-fa1454b035e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:52.077232 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70888f0-839b-48bb-a2f5-fa1454b035e3 None admin] PATH_INFO: `/v3/users/7b70d725813d4e5aa8690d4d7d37d9ff` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:52.078393 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70888f0-839b-48bb-a2f5-fa1454b035e3 None admin] RBAC: Authorizing `identity:get_user(user_id=7b70d725813d4e5aa8690d4d7d37d9ff)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:52.079881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70888f0-839b-48bb-a2f5-fa1454b035e3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:52.083144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 128/252] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:51 2022] GET /identity/v3/users/7b70d725813d4e5aa8690d4d7d37d9ff => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:52.089871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70888f0-839b-48bb-a2f5-fa1454b035e3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:52.140184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:52.156348 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d7126-f208-4d83-8cff-3f0d2cea2e10 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:52.198040 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70888f0-839b-48bb-a2f5-fa1454b035e3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:52.198632 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889a7c6-b89e-41c9-950a-7c26e43c4757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:52.199295 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889a7c6-b89e-41c9-950a-7c26e43c4757 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:52.199824 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889a7c6-b89e-41c9-950a-7c26e43c4757 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:52.200094 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889a7c6-b89e-41c9-950a-7c26e43c4757 None admin] PATH_INFO: `/v3/users/0dbaa80f37d34d028ab3e6f0a4fd25ec` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:52.202131 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889a7c6-b89e-41c9-950a-7c26e43c4757 None admin] RBAC: Authorizing `identity:get_user(user_id=0dbaa80f37d34d028ab3e6f0a4fd25ec)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:52.203606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889a7c6-b89e-41c9-950a-7c26e43c4757 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:52.206123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 125/253] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:52 2022] GET /identity/v3/users/0dbaa80f37d34d028ab3e6f0a4fd25ec => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:52.213308 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889a7c6-b89e-41c9-950a-7c26e43c4757 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:52.245118 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabde66c-7959-4372-a9f3-1fe898102cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:52.245897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabde66c-7959-4372-a9f3-1fe898102cd2 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:52.246177 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabde66c-7959-4372-a9f3-1fe898102cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:52.246457 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabde66c-7959-4372-a9f3-1fe898102cd2 None admin] PATH_INFO: `/v3/system/users/7b70d725813d4e5aa8690d4d7d37d9ff/roles/062b4d142bbc4085acd6efc13e6b2b74` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:52.247991 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabde66c-7959-4372-a9f3-1fe898102cd2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7b70d725813d4e5aa8690d4d7d37d9ff, role_id=062b4d142bbc4085acd6efc13e6b2b74)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:52.249880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabde66c-7959-4372-a9f3-1fe898102cd2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:52.259627 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 129/254] 10.222.0.18 () {58 vars in 1334 bytes} [Fri Oct 14 02:11:52 2022] PUT /identity/v3/system/users/7b70d725813d4e5aa8690d4d7d37d9ff/roles/062b4d142bbc4085acd6efc13e6b2b74 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:11:52.290057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:52.343014 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889a7c6-b89e-41c9-950a-7c26e43c4757 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:52.384027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3488b2-6feb-48c1-8a37-dd74ab17da6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:52.385038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3488b2-6feb-48c1-8a37-dd74ab17da6c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:52.385508 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3488b2-6feb-48c1-8a37-dd74ab17da6c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:52.385999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3488b2-6feb-48c1-8a37-dd74ab17da6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:52.386968 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3488b2-6feb-48c1-8a37-dd74ab17da6c None admin] RBAC: Adding query filter params (role.id=e460c7cf7b5b4c35a5a0e326f69a9fe3, user.id=0dbaa80f37d34d028ab3e6f0a4fd25ec, scope.system=all) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:52.387522 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3488b2-6feb-48c1-8a37-dd74ab17da6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:52.389185 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3488b2-6feb-48c1-8a37-dd74ab17da6c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:52.403152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 126/255] 10.222.0.18 () {56 vars in 1308 bytes} [Fri Oct 14 02:11:52 2022] GET /identity/v3/role_assignments?user.id=0dbaa80f37d34d028ab3e6f0a4fd25ec&scope.system=all&role.id=e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:54.285408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eade6-63e0-418d-a213-41dca13f7561 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:54.286021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eade6-63e0-418d-a213-41dca13f7561 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:54.286021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eade6-63e0-418d-a213-41dca13f7561 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:54.287492 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 130/256] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:54.292881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fa151-4528-4604-8554-1b9749e10fc6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:54.293679 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fa151-4528-4604-8554-1b9749e10fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:54.294162 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fa151-4528-4604-8554-1b9749e10fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:54.321413 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d36fa151-4528-4604-8554-1b9749e10fc6 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:54.322983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d36fa151-4528-4604-8554-1b9749e10fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:54.337695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330ca6f-92d5-4b1f-8d98-192220c78188 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:54.338003 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330ca6f-92d5-4b1f-8d98-192220c78188 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:54.338241 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330ca6f-92d5-4b1f-8d98-192220c78188 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:54.340333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 131/257] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:54.346407 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ed403-b02e-4059-baea-4c6925b298f7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:54.346694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ed403-b02e-4059-baea-4c6925b298f7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:54.346958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ed403-b02e-4059-baea-4c6925b298f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:54.384267 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b62ed403-b02e-4059-baea-4c6925b298f7 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:54.385513 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b62ed403-b02e-4059-baea-4c6925b298f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:54.420702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 127/258] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 14 02:11:54.479091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 132/259] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 14 02:11:55.005373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fa151-4528-4604-8554-1b9749e10fc6 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:55.018680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fa151-4528-4604-8554-1b9749e10fc6 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:55.023303 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bbd1f-4b61-44f6-ba4b-0cd0f5e9e760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:55.024226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bbd1f-4b61-44f6-ba4b-0cd0f5e9e760 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:55.025200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ed403-b02e-4059-baea-4c6925b298f7 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:55.030705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bbd1f-4b61-44f6-ba4b-0cd0f5e9e760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:55.030998 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bbd1f-4b61-44f6-ba4b-0cd0f5e9e760 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:55.034982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ed403-b02e-4059-baea-4c6925b298f7 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:55.046563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b3989-0fba-407a-9291-50a6b12de0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:55.047331 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b3989-0fba-407a-9291-50a6b12de0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:55.047612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b3989-0fba-407a-9291-50a6b12de0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:55.047886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b3989-0fba-407a-9291-50a6b12de0b1 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:55.061869 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bbd1f-4b61-44f6-ba4b-0cd0f5e9e760 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-809bbd1f-4b61-44f6-ba4b-0cd0f5e9e760 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:55.067382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:55.068319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:55.069057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:55.069057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:55.069057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:55.069057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:55.069057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:55.069057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:55.069057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:55.069057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:55.069057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:55.069057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:55.069057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:55.069057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:55.076533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 128/260] 10.222.0.18 () {56 vars in 1120 bytes} [Fri Oct 14 02:11:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 66 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:55.080975 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf95d83-93b2-41fd-9434-ed451f7a03cf None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:55.081291 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf95d83-93b2-41fd-9434-ed451f7a03cf None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:55.082115 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf95d83-93b2-41fd-9434-ed451f7a03cf None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:55.083627 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 129/261] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:55.084913 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42b3989-0fba-407a-9291-50a6b12de0b1 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-a42b3989-0fba-407a-9291-50a6b12de0b1 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:55.091246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:55.093563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:55.095465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:55.095465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:55.095465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:55.095465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:55.095465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:55.095465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:55.095465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:55.095465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:55.095465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:55.095465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:55.095465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:11:55.095465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:55.095465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 133/262] 10.222.0.18 () {56 vars in 1135 bytes} [Fri Oct 14 02:11:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 68 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:11:55.097820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf95d83-93b2-41fd-9434-ed451f7a03cf None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:55.097959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb5dbd-de3f-4cbf-ab8a-acc7ef31cdce None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:55.097959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb5dbd-de3f-4cbf-ab8a-acc7ef31cdce None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:55.097959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb5dbd-de3f-4cbf-ab8a-acc7ef31cdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:55.117643 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf95d83-93b2-41fd-9434-ed451f7a03cf None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:55.125428 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-dbdb5dbd-de3f-4cbf-ab8a-acc7ef31cdce None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:55.126184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435b7ef-4643-4dab-b1ba-db48874844d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:55.126702 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dbdb5dbd-de3f-4cbf-ab8a-acc7ef31cdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:55.127324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435b7ef-4643-4dab-b1ba-db48874844d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:55.127791 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435b7ef-4643-4dab-b1ba-db48874844d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:55.128248 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435b7ef-4643-4dab-b1ba-db48874844d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:55.129132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435b7ef-4643-4dab-b1ba-db48874844d4 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:55.129646 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435b7ef-4643-4dab-b1ba-db48874844d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:55.136893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-c435b7ef-4643-4dab-b1ba-db48874844d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:11:55.136893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 14 02:11:55.136893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: Oct 14 02:11:55.136893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435b7ef-4643-4dab-b1ba-db48874844d4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:55.150288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 130/263] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Oct 14 02:11:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:55.152408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435b7ef-4643-4dab-b1ba-db48874844d4 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:55.171742 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435b7ef-4643-4dab-b1ba-db48874844d4 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:55.183182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae4297-8e54-4296-92c0-0d8bae3660bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:55.184752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae4297-8e54-4296-92c0-0d8bae3660bd admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:55.185236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae4297-8e54-4296-92c0-0d8bae3660bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:55.190526 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae4297-8e54-4296-92c0-0d8bae3660bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:55.191472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae4297-8e54-4296-92c0-0d8bae3660bd admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:55.191958 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae4297-8e54-4296-92c0-0d8bae3660bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:55.193756 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae4297-8e54-4296-92c0-0d8bae3660bd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:55.196101 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:55.209686 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 131/264] 10.222.0.18 () {56 vars in 1132 bytes} [Fri Oct 14 02:11:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:55.212027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae4297-8e54-4296-92c0-0d8bae3660bd admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:55.230345 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae4297-8e54-4296-92c0-0d8bae3660bd admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:55.239424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed50a3d-e32b-437c-9dc3-c9ab5cc87271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:55.240230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed50a3d-e32b-437c-9dc3-c9ab5cc87271 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:55.240508 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed50a3d-e32b-437c-9dc3-c9ab5cc87271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:55.240785 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed50a3d-e32b-437c-9dc3-c9ab5cc87271 admin admin] PATH_INFO: `/v3/groups/3165bc6806024028b3f922fef1cdc82d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:55.246384 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 134/265] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:55.247573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb5dbd-de3f-4cbf-ab8a-acc7ef31cdce None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:55.248406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed50a3d-e32b-437c-9dc3-c9ab5cc87271 admin admin] RBAC: Authorizing `identity:get_group(group_id=3165bc6806024028b3f922fef1cdc82d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:55.249600 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-bed50a3d-e32b-437c-9dc3-c9ab5cc87271 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:11:55.249600 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 14 02:11:55.249600 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: Oct 14 02:11:55.250919 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed50a3d-e32b-437c-9dc3-c9ab5cc87271 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:55.252818 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 132/266] 10.222.0.18 () {56 vars in 1201 bytes} [Fri Oct 14 02:11:55 2022] GET /identity/v3/groups/3165bc6806024028b3f922fef1cdc82d => generated 226 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:55.254429 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6750d-b9d0-4cc9-aefa-8b743abdf6f5 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:55.254780 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6750d-b9d0-4cc9-aefa-8b743abdf6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:55.259230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6750d-b9d0-4cc9-aefa-8b743abdf6f5 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:55.262086 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 133/267] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:55.263320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6750d-b9d0-4cc9-aefa-8b743abdf6f5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:55.268068 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb5dbd-de3f-4cbf-ab8a-acc7ef31cdce None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:55.273436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96142e2c-9108-4afe-86cb-f2ce44186a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:55.274229 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96142e2c-9108-4afe-86cb-f2ce44186a5d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:55.274491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96142e2c-9108-4afe-86cb-f2ce44186a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:55.274833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96142e2c-9108-4afe-86cb-f2ce44186a5d admin admin] PATH_INFO: `/v3/groups/3165bc6806024028b3f922fef1cdc82d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:55.278555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96142e2c-9108-4afe-86cb-f2ce44186a5d admin admin] RBAC: Authorizing `identity:get_group(group_id=3165bc6806024028b3f922fef1cdc82d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:55.280579 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-96142e2c-9108-4afe-86cb-f2ce44186a5d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:11:55.280579 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 14 02:11:55.280579 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: Oct 14 02:11:55.281957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96142e2c-9108-4afe-86cb-f2ce44186a5d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:55.283665 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6750d-b9d0-4cc9-aefa-8b743abdf6f5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:55.284154 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 135/268] 10.222.0.18 () {56 vars in 1201 bytes} [Fri Oct 14 02:11:55 2022] GET /identity/v3/groups/3165bc6806024028b3f922fef1cdc82d => generated 226 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:55.285454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2c868-659c-4edd-8ee7-604455078ac7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:55.285973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2c868-659c-4edd-8ee7-604455078ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:55.285973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2c868-659c-4edd-8ee7-604455078ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:55.292136 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451ccb8-a76d-4db4-805a-3144c3397a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:55.293114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451ccb8-a76d-4db4-805a-3144c3397a02 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:55.293953 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451ccb8-a76d-4db4-805a-3144c3397a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:55.294486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451ccb8-a76d-4db4-805a-3144c3397a02 admin admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:55.295924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451ccb8-a76d-4db4-805a-3144c3397a02 admin admin] RBAC: Authorizing `identity:get_project(project_id=dabc91be3a7240e8bab6128f948854f0)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:55.298297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451ccb8-a76d-4db4-805a-3144c3397a02 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:55.301001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 134/269] 10.222.0.18 () {56 vars in 1207 bytes} [Fri Oct 14 02:11:55 2022] GET /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0 => generated 298 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:55.301937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451ccb8-a76d-4db4-805a-3144c3397a02 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:55.312514 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451ccb8-a76d-4db4-805a-3144c3397a02 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:55.315670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-71a2c868-659c-4edd-8ee7-604455078ac7 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:55.317079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-71a2c868-659c-4edd-8ee7-604455078ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:55.318213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba458e-7c43-42d2-a74b-65870b2c56e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:55.319112 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba458e-7c43-42d2-a74b-65870b2c56e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:55.319526 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba458e-7c43-42d2-a74b-65870b2c56e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:55.319873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba458e-7c43-42d2-a74b-65870b2c56e5 admin admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:55.321314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba458e-7c43-42d2-a74b-65870b2c56e5 admin admin] RBAC: Authorizing `identity:get_project(project_id=dabc91be3a7240e8bab6128f948854f0)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:55.323527 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba458e-7c43-42d2-a74b-65870b2c56e5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:55.325651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 136/271] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 14 02:11:55.326424 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2c868-659c-4edd-8ee7-604455078ac7 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:55.327945 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 135/271] 10.222.0.18 () {56 vars in 1207 bytes} [Fri Oct 14 02:11:55 2022] GET /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0 => generated 298 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:55.332075 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba458e-7c43-42d2-a74b-65870b2c56e5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:55.340973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2c868-659c-4edd-8ee7-604455078ac7 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:55.341996 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba458e-7c43-42d2-a74b-65870b2c56e5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:55.353757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afff7f-9ddf-4759-9dc7-96ab0f92dced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:55.354482 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afff7f-9ddf-4759-9dc7-96ab0f92dced admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:55.354759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afff7f-9ddf-4759-9dc7-96ab0f92dced admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:55.354878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af215e69-5cf6-4e79-b409-0d091844f720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:55.355105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afff7f-9ddf-4759-9dc7-96ab0f92dced admin admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0/groups/3165bc6806024028b3f922fef1cdc82d/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:55.355563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af215e69-5cf6-4e79-b409-0d091844f720 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:55.355933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af215e69-5cf6-4e79-b409-0d091844f720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:55.356217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af215e69-5cf6-4e79-b409-0d091844f720 admin admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0/groups/3165bc6806024028b3f922fef1cdc82d/roles/a3287688078248ecb19a6d9c85839631` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:55.358633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84afff7f-9ddf-4759-9dc7-96ab0f92dced admin admin] RBAC: Authorizing `identity:create_grant(project_id=dabc91be3a7240e8bab6128f948854f0, group_id=3165bc6806024028b3f922fef1cdc82d, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:55.359680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-84afff7f-9ddf-4759-9dc7-96ab0f92dced admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:11:55.359680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 14 02:11:55.359680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: Oct 14 02:11:55.362779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84afff7f-9ddf-4759-9dc7-96ab0f92dced admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:55.363651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af215e69-5cf6-4e79-b409-0d091844f720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dabc91be3a7240e8bab6128f948854f0, group_id=3165bc6806024028b3f922fef1cdc82d, role_id=a3287688078248ecb19a6d9c85839631)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:55.364743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-af215e69-5cf6-4e79-b409-0d091844f720 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:11:55.364743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 14 02:11:55.364743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: Oct 14 02:11:55.366944 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af215e69-5cf6-4e79-b409-0d091844f720 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:55.385476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:55.386135 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 137/272] 10.222.0.18 () {58 vars in 1463 bytes} [Fri Oct 14 02:11:55 2022] PUT /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0/groups/3165bc6806024028b3f922fef1cdc82d/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:11:55.395579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 136/273] 10.222.0.18 () {58 vars in 1463 bytes} [Fri Oct 14 02:11:55 2022] PUT /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0/groups/3165bc6806024028b3f922fef1cdc82d/roles/a3287688078248ecb19a6d9c85839631 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:11:55.818285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afff7f-9ddf-4759-9dc7-96ab0f92dced admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:55.872793 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af215e69-5cf6-4e79-b409-0d091844f720 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:55.961607 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afff7f-9ddf-4759-9dc7-96ab0f92dced admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:56.000102 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af215e69-5cf6-4e79-b409-0d091844f720 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:56.011116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5b720-2a24-48e2-ba6b-d17eb19c1e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:56.011778 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5b720-2a24-48e2-ba6b-d17eb19c1e77 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:56.012058 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5b720-2a24-48e2-ba6b-d17eb19c1e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:56.012333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5b720-2a24-48e2-ba6b-d17eb19c1e77 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:56.041827 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84b6d6-a242-4ce7-8b94-385ba40219de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:56.042837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84b6d6-a242-4ce7-8b94-385ba40219de None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:56.043143 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84b6d6-a242-4ce7-8b94-385ba40219de None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:56.043913 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5b720-2a24-48e2-ba6b-d17eb19c1e77 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:56.045791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84b6d6-a242-4ce7-8b94-385ba40219de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-91c5b720-2a24-48e2-ba6b-d17eb19c1e77 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:56.057856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:56.058783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:56.059587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:56.059587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:56.059587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:56.059587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:56.059587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:56.059587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:56.059587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:56.059587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:56.059587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:56.059587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:56.059587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:56.059587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:56.062594 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 137/274] 10.222.0.18 () {56 vars in 1120 bytes} [Fri Oct 14 02:11:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:11:56.072157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84b6d6-a242-4ce7-8b94-385ba40219de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-7b84b6d6-a242-4ce7-8b94-385ba40219de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:56.077275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:56.079597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:56.082397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:56.082397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:56.082397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:56.082397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:56.082397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:56.082397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:56.082397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:56.082397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:56.082397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:56.082397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:56.082397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:11:56.082397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:56.082397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 138/275] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:56.088090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5b720-2a24-48e2-ba6b-d17eb19c1e77 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:56.097318 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5b720-2a24-48e2-ba6b-d17eb19c1e77 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:56.099879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84b6d6-a242-4ce7-8b94-385ba40219de None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:56.115783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442323ba-2c54-4318-a9eb-75b380702890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:56.116454 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442323ba-2c54-4318-a9eb-75b380702890 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:56.119105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442323ba-2c54-4318-a9eb-75b380702890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:56.119391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442323ba-2c54-4318-a9eb-75b380702890 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:56.120078 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442323ba-2c54-4318-a9eb-75b380702890 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:56.121553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442323ba-2c54-4318-a9eb-75b380702890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:56.126456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442323ba-2c54-4318-a9eb-75b380702890 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:56.140784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:56.154152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 138/276] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Oct 14 02:11:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:56.164512 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442323ba-2c54-4318-a9eb-75b380702890 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:56.184046 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442323ba-2c54-4318-a9eb-75b380702890 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:56.194258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed977a0-05f6-416f-bf14-5449ee2ab783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:56.200110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed977a0-05f6-416f-bf14-5449ee2ab783 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:56.200405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed977a0-05f6-416f-bf14-5449ee2ab783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:56.201506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed977a0-05f6-416f-bf14-5449ee2ab783 admin admin] PATH_INFO: `/v3/groups/3165bc6806024028b3f922fef1cdc82d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:56.204174 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed977a0-05f6-416f-bf14-5449ee2ab783 admin admin] RBAC: Authorizing `identity:get_group(group_id=3165bc6806024028b3f922fef1cdc82d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:56.208627 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed977a0-05f6-416f-bf14-5449ee2ab783 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:56.211842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 139/277] 10.222.0.18 () {56 vars in 1201 bytes} [Fri Oct 14 02:11:56 2022] GET /identity/v3/groups/3165bc6806024028b3f922fef1cdc82d => generated 226 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:56.224571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed977a0-05f6-416f-bf14-5449ee2ab783 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:56.244081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed977a0-05f6-416f-bf14-5449ee2ab783 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:56.254060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb2de7-0fca-4c68-ab37-64c32069ea8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:56.259511 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb2de7-0fca-4c68-ab37-64c32069ea8a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:56.259813 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb2de7-0fca-4c68-ab37-64c32069ea8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:56.260097 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb2de7-0fca-4c68-ab37-64c32069ea8a admin admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:56.262650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb2de7-0fca-4c68-ab37-64c32069ea8a admin admin] RBAC: Authorizing `identity:get_project(project_id=f94fce41714c4f3099c1ff9eb62caf49)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:56.266568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb2de7-0fca-4c68-ab37-64c32069ea8a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:56.273333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 140/278] 10.222.0.18 () {56 vars in 1207 bytes} [Fri Oct 14 02:11:56 2022] GET /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49 => generated 302 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:56.282155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb2de7-0fca-4c68-ab37-64c32069ea8a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:56.299348 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb2de7-0fca-4c68-ab37-64c32069ea8a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:56.312179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d1b88-26a8-453b-92da-db1abe4295fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:56.314457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d1b88-26a8-453b-92da-db1abe4295fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:56.314849 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d1b88-26a8-453b-92da-db1abe4295fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:56.315153 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d1b88-26a8-453b-92da-db1abe4295fb admin admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49/groups/3165bc6806024028b3f922fef1cdc82d/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:56.318967 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d1b88-26a8-453b-92da-db1abe4295fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f94fce41714c4f3099c1ff9eb62caf49, group_id=3165bc6806024028b3f922fef1cdc82d, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:56.321908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d1b88-26a8-453b-92da-db1abe4295fb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:56.352319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 141/279] 10.222.0.18 () {58 vars in 1463 bytes} [Fri Oct 14 02:11:56 2022] PUT /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49/groups/3165bc6806024028b3f922fef1cdc82d/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:11:56.353607 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc3237-1e84-4a25-af42-0de6954e22d7 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:56.353892 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc3237-1e84-4a25-af42-0de6954e22d7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:56.354168 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc3237-1e84-4a25-af42-0de6954e22d7 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:56.356817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 142/280] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:56.361223 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377bcb-607a-4702-95b4-456dcb4ec45a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:56.361600 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377bcb-607a-4702-95b4-456dcb4ec45a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:56.361830 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377bcb-607a-4702-95b4-456dcb4ec45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:56.365019 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84b6d6-a242-4ce7-8b94-385ba40219de None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:56.398315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0d377bcb-607a-4702-95b4-456dcb4ec45a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:56.399571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0d377bcb-607a-4702-95b4-456dcb4ec45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:56.450271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2dc9fe-8b3d-484a-9f91-80de3e6b50ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:56.450992 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dc9fe-8b3d-484a-9f91-80de3e6b50ac None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:56.455437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dc9fe-8b3d-484a-9f91-80de3e6b50ac None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:56.455797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dc9fe-8b3d-484a-9f91-80de3e6b50ac None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:56.456482 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2dc9fe-8b3d-484a-9f91-80de3e6b50ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:56.456797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2dc9fe-8b3d-484a-9f91-80de3e6b50ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:56.458247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2dc9fe-8b3d-484a-9f91-80de3e6b50ac None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:56.482951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 139/281] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:56.489475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2dc9fe-8b3d-484a-9f91-80de3e6b50ac None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:56.561858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 143/282] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 14 02:11:56.631019 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:56.667350 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2dc9fe-8b3d-484a-9f91-80de3e6b50ac None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:56.712604 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a119c-255c-4a5b-9a3c-7a61d469955c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:56.713247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a119c-255c-4a5b-9a3c-7a61d469955c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:56.713525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a119c-255c-4a5b-9a3c-7a61d469955c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:56.713793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a119c-255c-4a5b-9a3c-7a61d469955c None admin] PATH_INFO: `/v3/users/7b70d725813d4e5aa8690d4d7d37d9ff` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:56.714936 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a119c-255c-4a5b-9a3c-7a61d469955c None admin] RBAC: Authorizing `identity:get_user(user_id=7b70d725813d4e5aa8690d4d7d37d9ff)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:56.716395 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a119c-255c-4a5b-9a3c-7a61d469955c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:56.718278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 140/283] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:11:56 2022] GET /identity/v3/users/7b70d725813d4e5aa8690d4d7d37d9ff => generated 293 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:11:56.729305 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d377bcb-607a-4702-95b4-456dcb4ec45a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:56.813899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c916cc-abf4-4de2-8c77-d42f3c743420 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:56.814065 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c916cc-abf4-4de2-8c77-d42f3c743420 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:56.814460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c916cc-abf4-4de2-8c77-d42f3c743420 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:56.815700 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:56.816129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 141/284] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:56.827547 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb01ba-8dc0-406f-a480-83a413b7bb4c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:56.827918 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb01ba-8dc0-406f-a480-83a413b7bb4c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:56.828459 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb01ba-8dc0-406f-a480-83a413b7bb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:56.869540 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-2fcb01ba-8dc0-406f-a480-83a413b7bb4c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:56.870767 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2fcb01ba-8dc0-406f-a480-83a413b7bb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:56.889792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 142/285] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 14 02:11:56.908596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d377bcb-607a-4702-95b4-456dcb4ec45a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:56.957885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e153fa-f061-4c0d-bfe3-45082ccdb59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:56.958521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e153fa-f061-4c0d-bfe3-45082ccdb59c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:56.958795 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e153fa-f061-4c0d-bfe3-45082ccdb59c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:56.959078 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e153fa-f061-4c0d-bfe3-45082ccdb59c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:56.959850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e153fa-f061-4c0d-bfe3-45082ccdb59c None admin] RBAC: Adding query filter params (role.id=062b4d142bbc4085acd6efc13e6b2b74, user.id=7b70d725813d4e5aa8690d4d7d37d9ff, scope.system=all) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:56.960154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e153fa-f061-4c0d-bfe3-45082ccdb59c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:56.961650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e153fa-f061-4c0d-bfe3-45082ccdb59c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:56.979733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 144/286] 10.222.0.18 () {56 vars in 1308 bytes} [Fri Oct 14 02:11:56 2022] GET /identity/v3/role_assignments?user.id=7b70d725813d4e5aa8690d4d7d37d9ff&scope.system=all&role.id=062b4d142bbc4085acd6efc13e6b2b74 => generated 510 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:57.110267 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb01ba-8dc0-406f-a480-83a413b7bb4c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:57.119964 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb01ba-8dc0-406f-a480-83a413b7bb4c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:57.128178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62061fab-c2fc-48cf-9acf-2778da778c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:57.128832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62061fab-c2fc-48cf-9acf-2778da778c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:57.129108 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62061fab-c2fc-48cf-9acf-2778da778c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:57.129372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62061fab-c2fc-48cf-9acf-2778da778c88 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:57.144043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62061fab-c2fc-48cf-9acf-2778da778c88 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-62061fab-c2fc-48cf-9acf-2778da778c88 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:57.148975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:57.149895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:57.151324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:57.151324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:57.151324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:57.151324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:57.151324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:57.151324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:57.151324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:57.151324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:57.151324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:57.151324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:57.151324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:11:57.151324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:57.151324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 143/287] 10.222.0.18 () {56 vars in 1135 bytes} [Fri Oct 14 02:11:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:57.155987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e153fa-f061-4c0d-bfe3-45082ccdb59c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:57.172060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e153fa-f061-4c0d-bfe3-45082ccdb59c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:57.176888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0dbfd2-b5d6-404d-8f0e-40f9cb4ea355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:57.177503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dbfd2-b5d6-404d-8f0e-40f9cb4ea355 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:57.177774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dbfd2-b5d6-404d-8f0e-40f9cb4ea355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:57.178040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dbfd2-b5d6-404d-8f0e-40f9cb4ea355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:57.178714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0dbfd2-b5d6-404d-8f0e-40f9cb4ea355 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:57.179012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0dbfd2-b5d6-404d-8f0e-40f9cb4ea355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:57.180637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0dbfd2-b5d6-404d-8f0e-40f9cb4ea355 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:57.197280 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 145/288] 10.222.0.18 () {56 vars in 1132 bytes} [Fri Oct 14 02:11:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:57.202658 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62061fab-c2fc-48cf-9acf-2778da778c88 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:57.216560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62061fab-c2fc-48cf-9acf-2778da778c88 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:57.224306 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aaa8ce-5334-490e-9dd0-a5757f9a3f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:57.225010 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aaa8ce-5334-490e-9dd0-a5757f9a3f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:57.225289 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aaa8ce-5334-490e-9dd0-a5757f9a3f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:57.225559 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aaa8ce-5334-490e-9dd0-a5757f9a3f9a admin admin] PATH_INFO: `/v3/groups/3165bc6806024028b3f922fef1cdc82d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:57.228013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aaa8ce-5334-490e-9dd0-a5757f9a3f9a admin admin] RBAC: Authorizing `identity:get_group(group_id=3165bc6806024028b3f922fef1cdc82d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:57.230146 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aaa8ce-5334-490e-9dd0-a5757f9a3f9a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:57.232237 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 144/289] 10.222.0.18 () {56 vars in 1201 bytes} [Fri Oct 14 02:11:57 2022] GET /identity/v3/groups/3165bc6806024028b3f922fef1cdc82d => generated 226 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:57.238331 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0dbfd2-b5d6-404d-8f0e-40f9cb4ea355 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:57.251599 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0dbfd2-b5d6-404d-8f0e-40f9cb4ea355 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:57.259233 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90434aea-a266-42ab-bf95-32c1ef148fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:57.259885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90434aea-a266-42ab-bf95-32c1ef148fca admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:57.260162 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90434aea-a266-42ab-bf95-32c1ef148fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:57.260432 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90434aea-a266-42ab-bf95-32c1ef148fca admin admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:57.261570 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90434aea-a266-42ab-bf95-32c1ef148fca admin admin] RBAC: Authorizing `identity:get_project(project_id=f94fce41714c4f3099c1ff9eb62caf49)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:57.263758 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90434aea-a266-42ab-bf95-32c1ef148fca admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:57.265708 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 146/290] 10.222.0.18 () {56 vars in 1207 bytes} [Fri Oct 14 02:11:57 2022] GET /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49 => generated 302 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:57.272133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aaa8ce-5334-490e-9dd0-a5757f9a3f9a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:57.284388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aaa8ce-5334-490e-9dd0-a5757f9a3f9a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:57.293626 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90434aea-a266-42ab-bf95-32c1ef148fca admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:57.296038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840a80c-12c0-4eba-a43d-32fab34d84f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:57.296780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a80c-12c0-4eba-a43d-32fab34d84f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:57.297061 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a80c-12c0-4eba-a43d-32fab34d84f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:57.297333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a80c-12c0-4eba-a43d-32fab34d84f5 admin admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49/groups/3165bc6806024028b3f922fef1cdc82d/roles/a3287688078248ecb19a6d9c85839631` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:57.300810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840a80c-12c0-4eba-a43d-32fab34d84f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f94fce41714c4f3099c1ff9eb62caf49, group_id=3165bc6806024028b3f922fef1cdc82d, role_id=a3287688078248ecb19a6d9c85839631)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:57.303566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840a80c-12c0-4eba-a43d-32fab34d84f5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:57.308879 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90434aea-a266-42ab-bf95-32c1ef148fca admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:57.317999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573b5d9-4022-4e2c-a9e7-7abe551ee6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:57.318707 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573b5d9-4022-4e2c-a9e7-7abe551ee6dc admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:57.318990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573b5d9-4022-4e2c-a9e7-7abe551ee6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:57.319288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573b5d9-4022-4e2c-a9e7-7abe551ee6dc admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:57.333409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 145/291] 10.222.0.18 () {58 vars in 1463 bytes} [Fri Oct 14 02:11:57 2022] PUT /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49/groups/3165bc6806024028b3f922fef1cdc82d/roles/a3287688078248ecb19a6d9c85839631 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:11:57.337739 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573b5d9-4022-4e2c-a9e7-7abe551ee6dc admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-2573b5d9-4022-4e2c-a9e7-7abe551ee6dc admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:57.342831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:57.345040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:57.347179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:57.347179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:57.347179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:57.347179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:57.347179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:57.347179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:57.347179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:57.347179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:57.347179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:57.347179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:57.347179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:11:57.347179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:11:57.348653 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 147/292] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Oct 14 02:11:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:11:57.354386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840a80c-12c0-4eba-a43d-32fab34d84f5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:57.448635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840a80c-12c0-4eba-a43d-32fab34d84f5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:57.453134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a0a65-f6e7-4d68-8956-49cb87cc9f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:57.453752 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a0a65-f6e7-4d68-8956-49cb87cc9f73 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:57.454039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a0a65-f6e7-4d68-8956-49cb87cc9f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:57.454306 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a0a65-f6e7-4d68-8956-49cb87cc9f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:57.455692 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a0a65-f6e7-4d68-8956-49cb87cc9f73 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:57.456186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a0a65-f6e7-4d68-8956-49cb87cc9f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:57.457347 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-f91a0a65-f6e7-4d68-8956-49cb87cc9f73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:11:57.457347 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 14 02:11:57.457347 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: Oct 14 02:11:57.458262 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a0a65-f6e7-4d68-8956-49cb87cc9f73 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:57.469999 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 146/293] 10.222.0.18 () {56 vars in 1120 bytes} [Fri Oct 14 02:11:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:57.475503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573b5d9-4022-4e2c-a9e7-7abe551ee6dc admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:57.494991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573b5d9-4022-4e2c-a9e7-7abe551ee6dc admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:57.505239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4d1d9-d4b5-49af-ba0d-d5d8819484ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:57.505865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4d1d9-d4b5-49af-ba0d-d5d8819484ed admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:57.506146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4d1d9-d4b5-49af-ba0d-d5d8819484ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:57.506415 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4d1d9-d4b5-49af-ba0d-d5d8819484ed admin admin] PATH_INFO: `/v3/groups/78c813fdca9a48db8990e38f688d2856` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:57.515330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4d1d9-d4b5-49af-ba0d-d5d8819484ed admin admin] RBAC: Authorizing `identity:get_group(group_id=78c813fdca9a48db8990e38f688d2856)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:57.517255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4d1d9-d4b5-49af-ba0d-d5d8819484ed admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:57.519844 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 148/294] 10.222.0.18 () {56 vars in 1201 bytes} [Fri Oct 14 02:11:57 2022] GET /identity/v3/groups/78c813fdca9a48db8990e38f688d2856 => generated 229 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:57.525178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a0a65-f6e7-4d68-8956-49cb87cc9f73 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:57.540764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a0a65-f6e7-4d68-8956-49cb87cc9f73 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:57.547894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f5dce-4747-46d2-a81d-cf39d5fe2239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:57.548540 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f5dce-4747-46d2-a81d-cf39d5fe2239 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:57.549501 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f5dce-4747-46d2-a81d-cf39d5fe2239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:57.549983 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f5dce-4747-46d2-a81d-cf39d5fe2239 admin admin] PATH_INFO: `/v3/projects/c1e7392d098a4e219827c4ae7f8117e2` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:57.555869 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f5dce-4747-46d2-a81d-cf39d5fe2239 admin admin] RBAC: Authorizing `identity:get_project(project_id=c1e7392d098a4e219827c4ae7f8117e2)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:57.557705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f5dce-4747-46d2-a81d-cf39d5fe2239 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:57.559748 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 147/295] 10.222.0.18 () {56 vars in 1207 bytes} [Fri Oct 14 02:11:57 2022] GET /identity/v3/projects/c1e7392d098a4e219827c4ae7f8117e2 => generated 344 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:57.565483 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4d1d9-d4b5-49af-ba0d-d5d8819484ed admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:57.581414 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4d1d9-d4b5-49af-ba0d-d5d8819484ed admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:57.596848 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3504b4d-db01-45ab-bd45-8994578017e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:57.597558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3504b4d-db01-45ab-bd45-8994578017e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:57.597837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3504b4d-db01-45ab-bd45-8994578017e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:57.598109 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3504b4d-db01-45ab-bd45-8994578017e6 admin admin] PATH_INFO: `/v3/projects/c1e7392d098a4e219827c4ae7f8117e2/groups/78c813fdca9a48db8990e38f688d2856/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:57.600160 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3504b4d-db01-45ab-bd45-8994578017e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e7392d098a4e219827c4ae7f8117e2, group_id=78c813fdca9a48db8990e38f688d2856, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:57.602832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3504b4d-db01-45ab-bd45-8994578017e6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:57.618762 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 149/296] 10.222.0.18 () {58 vars in 1463 bytes} [Fri Oct 14 02:11:57 2022] PUT /identity/v3/projects/c1e7392d098a4e219827c4ae7f8117e2/groups/78c813fdca9a48db8990e38f688d2856/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:11:58.562861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf54c1a-8018-45a4-9f39-2058851ee63d None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:58.563589 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf54c1a-8018-45a4-9f39-2058851ee63d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:58.563589 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf54c1a-8018-45a4-9f39-2058851ee63d None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:58.565873 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 148/297] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:58.570743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb1c8e-bfe6-4154-bdb0-15c12792576b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:58.571239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb1c8e-bfe6-4154-bdb0-15c12792576b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:58.571392 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb1c8e-bfe6-4154-bdb0-15c12792576b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:58.596844 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-addb1c8e-bfe6-4154-bdb0-15c12792576b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:58.598052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-addb1c8e-bfe6-4154-bdb0-15c12792576b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:58.674741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:58.680450 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 150/298] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:58.864556 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143bf66-b978-4188-b1e5-d94efd1a28ac None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:58.864869 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143bf66-b978-4188-b1e5-d94efd1a28ac None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:58.865056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143bf66-b978-4188-b1e5-d94efd1a28ac None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:58.866656 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 149/299] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:58.873584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6560b56-7dbb-4b68-932a-64de6afeadab None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:58.873865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6560b56-7dbb-4b68-932a-64de6afeadab None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:58.874142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6560b56-7dbb-4b68-932a-64de6afeadab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:58.900534 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d6560b56-7dbb-4b68-932a-64de6afeadab None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:58.902060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d6560b56-7dbb-4b68-932a-64de6afeadab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:59.001671 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 151/300] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:11:59.070917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143bf66-b978-4188-b1e5-d94efd1a28ac None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:59.133458 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:59.208726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143bf66-b978-4188-b1e5-d94efd1a28ac None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:59.255399 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca64ec-0249-4fb7-840d-6b1f01ad07ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:59.256043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca64ec-0249-4fb7-840d-6b1f01ad07ba None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:59.256321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca64ec-0249-4fb7-840d-6b1f01ad07ba None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:59.256595 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca64ec-0249-4fb7-840d-6b1f01ad07ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:59.268513 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca64ec-0249-4fb7-840d-6b1f01ad07ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-9dca64ec-0249-4fb7-840d-6b1f01ad07ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:59.273887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:59.274707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:59.275493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:59.275493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:59.275493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:59.275493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:59.275493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:59.275493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:59.275493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:59.275493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:59.275493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:59.275493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:59.275493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:11:59.275493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:59.279395 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 150/301] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:11:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:59.282416 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6560b56-7dbb-4b68-932a-64de6afeadab None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:59.414004 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6560b56-7dbb-4b68-932a-64de6afeadab None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:59.424628 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca64ec-0249-4fb7-840d-6b1f01ad07ba None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:59.484787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:59.504822 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45f82e-f9fd-4526-8644-a0170580f677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:59.505453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45f82e-f9fd-4526-8644-a0170580f677 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:59.505726 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45f82e-f9fd-4526-8644-a0170580f677 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:59.505995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45f82e-f9fd-4526-8644-a0170580f677 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:59.506657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45f82e-f9fd-4526-8644-a0170580f677 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:59.507017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45f82e-f9fd-4526-8644-a0170580f677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:59.510638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45f82e-f9fd-4526-8644-a0170580f677 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:59.525674 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 152/302] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:11:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:59.531581 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45f82e-f9fd-4526-8644-a0170580f677 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:59.615533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca64ec-0249-4fb7-840d-6b1f01ad07ba None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:59.624201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:59.662929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a16063-ccff-448d-adc8-888de627f888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:59.663620 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a16063-ccff-448d-adc8-888de627f888 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:59.663909 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a16063-ccff-448d-adc8-888de627f888 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:59.664721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a16063-ccff-448d-adc8-888de627f888 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:59.677311 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a16063-ccff-448d-adc8-888de627f888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-60a16063-ccff-448d-adc8-888de627f888 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:11:59.684225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:11:59.685081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:11:59.685839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:11:59.685839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:11:59.685839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:11:59.685839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:11:59.685839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:11:59.685839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:11:59.685839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:11:59.685839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:11:59.685839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:11:59.685839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:11:59.685839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:11:59.685839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:11:59.686408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 151/303] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:11:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:11:59.689649 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a38c98-9471-4e96-9af4-b2b3cc5bbaa6 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:59.689649 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a38c98-9471-4e96-9af4-b2b3cc5bbaa6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:59.689649 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a38c98-9471-4e96-9af4-b2b3cc5bbaa6 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:59.689918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45f82e-f9fd-4526-8644-a0170580f677 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:59.690316 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 152/304] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:11:59.691397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a38c98-9471-4e96-9af4-b2b3cc5bbaa6 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:11:59.739757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05957c2-4c74-4aa8-b394-e2136edb2624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:59.740064 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05957c2-4c74-4aa8-b394-e2136edb2624 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:59.743121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05957c2-4c74-4aa8-b394-e2136edb2624 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:59.743121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05957c2-4c74-4aa8-b394-e2136edb2624 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:59.820277 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:59.823205 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a38c98-9471-4e96-9af4-b2b3cc5bbaa6 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:11:59.835359 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05957c2-4c74-4aa8-b394-e2136edb2624 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:59.837345 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05957c2-4c74-4aa8-b394-e2136edb2624 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:59.879982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943eb7ec-a628-4162-93ba-f745a9cb1509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:11:59.880994 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eb7ec-a628-4162-93ba-f745a9cb1509 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:59.881469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eb7ec-a628-4162-93ba-f745a9cb1509 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:59.881935 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eb7ec-a628-4162-93ba-f745a9cb1509 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:59.882799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943eb7ec-a628-4162-93ba-f745a9cb1509 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:11:59.883345 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943eb7ec-a628-4162-93ba-f745a9cb1509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:11:59.884923 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943eb7ec-a628-4162-93ba-f745a9cb1509 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:11:59.895176 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 153/305] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:11:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:11:59.896753 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdb1e4-8866-4c85-8ecc-2e177cb447fc None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:11:59.897272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdb1e4-8866-4c85-8ecc-2e177cb447fc None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:11:59.897738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdb1e4-8866-4c85-8ecc-2e177cb447fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:11:59.922109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-cfcdb1e4-8866-4c85-8ecc-2e177cb447fc None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:11:59.923699 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cfcdb1e4-8866-4c85-8ecc-2e177cb447fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:11:59.951902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:11:59.979340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 153/306] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:11:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 14 02:11:59.980368 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05957c2-4c74-4aa8-b394-e2136edb2624 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:00.008134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 154/307] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:00.009390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdb1e4-8866-4c85-8ecc-2e177cb447fc None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:00.097086 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:00.108635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05957c2-4c74-4aa8-b394-e2136edb2624 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:00.141881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdb1e4-8866-4c85-8ecc-2e177cb447fc None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:00.149831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1655e-ebc3-4cea-b058-e4c11637225d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:00.150665 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1655e-ebc3-4cea-b058-e4c11637225d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:00.151103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1655e-ebc3-4cea-b058-e4c11637225d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:00.151506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1655e-ebc3-4cea-b058-e4c11637225d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:00.197846 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869daaf-2880-43e7-9efb-80ea0aa40c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:00.198840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869daaf-2880-43e7-9efb-80ea0aa40c91 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:00.199350 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869daaf-2880-43e7-9efb-80ea0aa40c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:00.199828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869daaf-2880-43e7-9efb-80ea0aa40c91 None admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:00.201159 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869daaf-2880-43e7-9efb-80ea0aa40c91 None admin] RBAC: Authorizing `identity:get_project(project_id=dabc91be3a7240e8bab6128f948854f0)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:00.202789 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869daaf-2880-43e7-9efb-80ea0aa40c91 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:00.205139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 155/308] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:12:00 2022] GET /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:00.206559 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f5e34-cc42-4a89-a668-f61070b0d2bc None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:00.207051 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f5e34-cc42-4a89-a668-f61070b0d2bc None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:00.207577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f5e34-cc42-4a89-a668-f61070b0d2bc None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:00.208967 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 156/309] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:00.211003 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f5e34-cc42-4a89-a668-f61070b0d2bc None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:00.232025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1655e-ebc3-4cea-b058-e4c11637225d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:00.233515 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1655e-ebc3-4cea-b058-e4c11637225d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:00.241311 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:00.343792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f5e34-cc42-4a89-a668-f61070b0d2bc None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:00.355814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:00.371052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 154/310] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:11:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:12:00.372546 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17d30e-9747-4526-99de-f6e15e8f6a20 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:00.372837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17d30e-9747-4526-99de-f6e15e8f6a20 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:00.373370 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17d30e-9747-4526-99de-f6e15e8f6a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:00.391132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7cf45-f04b-4125-9aac-fc1c88b97fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:00.391780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7cf45-f04b-4125-9aac-fc1c88b97fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:00.392058 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7cf45-f04b-4125-9aac-fc1c88b97fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:00.392865 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7cf45-f04b-4125-9aac-fc1c88b97fe5 None admin] PATH_INFO: `/v3/groups/3165bc6806024028b3f922fef1cdc82d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:00.393430 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4e17d30e-9747-4526-99de-f6e15e8f6a20 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:00.394252 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7cf45-f04b-4125-9aac-fc1c88b97fe5 None admin] RBAC: Authorizing `identity:get_group(group_id=3165bc6806024028b3f922fef1cdc82d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:00.394629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4e17d30e-9747-4526-99de-f6e15e8f6a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:00.395721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7cf45-f04b-4125-9aac-fc1c88b97fe5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:00.397310 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 157/311] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Oct 14 02:12:00 2022] GET /identity/v3/groups/3165bc6806024028b3f922fef1cdc82d => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:00.398075 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7cf45-f04b-4125-9aac-fc1c88b97fe5 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:00.476567 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 155/312] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:00.477154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17d30e-9747-4526-99de-f6e15e8f6a20 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:00.521854 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:00.531791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7cf45-f04b-4125-9aac-fc1c88b97fe5 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:00.577366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773d389-f256-4a96-a23a-bd3c166cc0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:00.578550 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773d389-f256-4a96-a23a-bd3c166cc0fe None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:00.578946 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773d389-f256-4a96-a23a-bd3c166cc0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:00.579278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773d389-f256-4a96-a23a-bd3c166cc0fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:00.588486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17d30e-9747-4526-99de-f6e15e8f6a20 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:00.596185 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773d389-f256-4a96-a23a-bd3c166cc0fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-b773d389-f256-4a96-a23a-bd3c166cc0fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:00.602654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:00.603639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:00.604406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:00.604406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:00.604406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:00.604406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:00.604406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:00.604406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:00.604406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:00.604406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:00.604406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:00.604406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:00.604406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:12:00.604406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:00.604406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 158/313] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:12:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:00.605863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773d389-f256-4a96-a23a-bd3c166cc0fe None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:00.621964 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:00.641384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3ef0e-4aab-46f4-9443-8e5e21f3d644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:00.642031 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3ef0e-4aab-46f4-9443-8e5e21f3d644 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:00.642752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3ef0e-4aab-46f4-9443-8e5e21f3d644 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:00.642867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3ef0e-4aab-46f4-9443-8e5e21f3d644 None admin] PATH_INFO: `/v3/projects/dabc91be3a7240e8bab6128f948854f0` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:00.644077 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3ef0e-4aab-46f4-9443-8e5e21f3d644 None admin] RBAC: Authorizing `identity:get_project(project_id=dabc91be3a7240e8bab6128f948854f0)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:00.645530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3ef0e-4aab-46f4-9443-8e5e21f3d644 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:00.647339 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 156/314] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:12:00 2022] GET /identity/v3/projects/dabc91be3a7240e8bab6128f948854f0 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:00.648055 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3ef0e-4aab-46f4-9443-8e5e21f3d644 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:00.758140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773d389-f256-4a96-a23a-bd3c166cc0fe None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:00.780710 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3ef0e-4aab-46f4-9443-8e5e21f3d644 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:00.804064 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e5d8f-ced6-445c-9dbe-fdffa023c544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:00.804683 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e5d8f-ced6-445c-9dbe-fdffa023c544 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:00.804966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e5d8f-ced6-445c-9dbe-fdffa023c544 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:00.805834 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e5d8f-ced6-445c-9dbe-fdffa023c544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:00.805994 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e5d8f-ced6-445c-9dbe-fdffa023c544 None admin] RBAC: Adding query filter params (group.id=3165bc6806024028b3f922fef1cdc82d, role.id=e460c7cf7b5b4c35a5a0e326f69a9fe3, scope.project.id=dabc91be3a7240e8bab6128f948854f0) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:00.806281 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e5d8f-ced6-445c-9dbe-fdffa023c544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:00.808042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e5d8f-ced6-445c-9dbe-fdffa023c544 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:00.815458 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 159/315] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Oct 14 02:12:00 2022] GET /identity/v3/role_assignments?group.id=3165bc6806024028b3f922fef1cdc82d&scope.project.id=dabc91be3a7240e8bab6128f948854f0&role.id=e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 611 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:00.816214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e5d8f-ced6-445c-9dbe-fdffa023c544 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:00.822487 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11e7e6-c918-4355-a292-05e8473cb863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:00.823157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11e7e6-c918-4355-a292-05e8473cb863 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:00.823440 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11e7e6-c918-4355-a292-05e8473cb863 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:00.823709 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11e7e6-c918-4355-a292-05e8473cb863 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:00.824395 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11e7e6-c918-4355-a292-05e8473cb863 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:00.824695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11e7e6-c918-4355-a292-05e8473cb863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:00.826784 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11e7e6-c918-4355-a292-05e8473cb863 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:00.837147 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 157/316] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:12:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:00.838255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:00.838365 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe2dc-99ef-4df6-832e-9be54d4416eb None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:00.838590 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe2dc-99ef-4df6-832e-9be54d4416eb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:00.838867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe2dc-99ef-4df6-832e-9be54d4416eb None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:00.840110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 158/317] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:00.840806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfe2dc-99ef-4df6-832e-9be54d4416eb None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:00.923346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:00.958273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e5d8f-ced6-445c-9dbe-fdffa023c544 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:00.999441 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfe2dc-99ef-4df6-832e-9be54d4416eb None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:01.010080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15438c-37e6-4f7a-a17d-df5b96087a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:01.010727 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15438c-37e6-4f7a-a17d-df5b96087a08 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:01.011010 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15438c-37e6-4f7a-a17d-df5b96087a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:01.011514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15438c-37e6-4f7a-a17d-df5b96087a08 None admin] PATH_INFO: `/v3/groups/3165bc6806024028b3f922fef1cdc82d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:01.012499 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c15438c-37e6-4f7a-a17d-df5b96087a08 None admin] RBAC: Authorizing `identity:get_group(group_id=3165bc6806024028b3f922fef1cdc82d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:01.014004 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c15438c-37e6-4f7a-a17d-df5b96087a08 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:01.015691 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 160/318] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Oct 14 02:12:00 2022] GET /identity/v3/groups/3165bc6806024028b3f922fef1cdc82d => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:01.016472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15438c-37e6-4f7a-a17d-df5b96087a08 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:01.049107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e19f1-cb59-4da7-ae5e-28b5edc48232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:01.049846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e19f1-cb59-4da7-ae5e-28b5edc48232 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:01.050198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e19f1-cb59-4da7-ae5e-28b5edc48232 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:01.050541 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e19f1-cb59-4da7-ae5e-28b5edc48232 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:01.066492 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e19f1-cb59-4da7-ae5e-28b5edc48232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-4c6e19f1-cb59-4da7-ae5e-28b5edc48232 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:01.071557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:01.072396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:01.073189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:01.073189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:01.073189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:01.073189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:01.073189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:01.073189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:01.073189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:01.073189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:01.073189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:01.073189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:01.073189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:12:01.073189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:01.073189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 159/319] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:12:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:01.074314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359d5c7-9055-498b-b7aa-df65e70b4bba None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:01.074991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359d5c7-9055-498b-b7aa-df65e70b4bba None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:01.074991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359d5c7-9055-498b-b7aa-df65e70b4bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:01.100547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5359d5c7-9055-498b-b7aa-df65e70b4bba None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:01.101733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5359d5c7-9055-498b-b7aa-df65e70b4bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:01.119684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:01.139042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15438c-37e6-4f7a-a17d-df5b96087a08 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:01.178912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 160/320] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:01.180254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359d5c7-9055-498b-b7aa-df65e70b4bba None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:01.184296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec7b14-2560-4a0a-aee5-6debffb62bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:01.185215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec7b14-2560-4a0a-aee5-6debffb62bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:01.185507 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec7b14-2560-4a0a-aee5-6debffb62bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:01.185780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec7b14-2560-4a0a-aee5-6debffb62bf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:01.235028 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:01.270563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec7b14-2560-4a0a-aee5-6debffb62bf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:01.271958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec7b14-2560-4a0a-aee5-6debffb62bf1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:01.318116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359d5c7-9055-498b-b7aa-df65e70b4bba None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:01.357845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1094aef-fa12-4692-8b87-0cb7adfb0d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:01.358461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1094aef-fa12-4692-8b87-0cb7adfb0d6c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:01.358739 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1094aef-fa12-4692-8b87-0cb7adfb0d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:01.359009 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1094aef-fa12-4692-8b87-0cb7adfb0d6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:01.359787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1094aef-fa12-4692-8b87-0cb7adfb0d6c None admin] RBAC: Adding query filter params (group.id=3165bc6806024028b3f922fef1cdc82d, role.id=a3287688078248ecb19a6d9c85839631, scope.project.id=dabc91be3a7240e8bab6128f948854f0) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:01.360095 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1094aef-fa12-4692-8b87-0cb7adfb0d6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:01.362061 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1094aef-fa12-4692-8b87-0cb7adfb0d6c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:01.368915 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 161/321] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Oct 14 02:12:01 2022] GET /identity/v3/role_assignments?group.id=3165bc6806024028b3f922fef1cdc82d&scope.project.id=dabc91be3a7240e8bab6128f948854f0&role.id=a3287688078248ecb19a6d9c85839631 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:01.369748 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1094aef-fa12-4692-8b87-0cb7adfb0d6c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:01.378882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:01.404118 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 161/322] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:12:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:12:01.410322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec7b14-2560-4a0a-aee5-6debffb62bf1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:01.505095 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1094aef-fa12-4692-8b87-0cb7adfb0d6c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:01.508635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:01.552483 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f6fc8-8e4b-47b9-8512-c6a3624378b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:01.553357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f6fc8-8e4b-47b9-8512-c6a3624378b3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:01.553700 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f6fc8-8e4b-47b9-8512-c6a3624378b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:01.554089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f6fc8-8e4b-47b9-8512-c6a3624378b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:01.554790 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f6fc8-8e4b-47b9-8512-c6a3624378b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:01.555118 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f6fc8-8e4b-47b9-8512-c6a3624378b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:01.557189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f6fc8-8e4b-47b9-8512-c6a3624378b3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:01.561742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec7b14-2560-4a0a-aee5-6debffb62bf1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:01.568173 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 162/323] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:01.568336 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f6fc8-8e4b-47b9-8512-c6a3624378b3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:01.605671 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2ca11-628c-4997-99da-363b5f250c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:01.606608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2ca11-628c-4997-99da-363b5f250c39 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:01.606891 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2ca11-628c-4997-99da-363b5f250c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:01.607224 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2ca11-628c-4997-99da-363b5f250c39 None admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:01.608377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2ca11-628c-4997-99da-363b5f250c39 None admin] RBAC: Authorizing `identity:get_project(project_id=f94fce41714c4f3099c1ff9eb62caf49)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:01.609831 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2ca11-628c-4997-99da-363b5f250c39 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:01.612239 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 162/324] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:12:01 2022] GET /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:01.612876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2ca11-628c-4997-99da-363b5f250c39 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:01.685608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:01.694319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f6fc8-8e4b-47b9-8512-c6a3624378b3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:01.735265 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2ca11-628c-4997-99da-363b5f250c39 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:01.738586 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d46eb8-04eb-4b53-a00b-647124d7b16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:01.739379 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d46eb8-04eb-4b53-a00b-647124d7b16a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:01.739676 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d46eb8-04eb-4b53-a00b-647124d7b16a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:01.739954 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d46eb8-04eb-4b53-a00b-647124d7b16a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:01.754790 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d46eb8-04eb-4b53-a00b-647124d7b16a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-c0d46eb8-04eb-4b53-a00b-647124d7b16a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:01.760204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:01.761149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:01.761921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:01.761921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:01.761921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:01.761921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:01.761921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:01.761921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:01.761921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:01.761921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:01.761921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:01.761921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:01.761921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:12:01.761921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:01.761921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 163/325] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:12:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:01.762555 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d46eb8-04eb-4b53-a00b-647124d7b16a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:01.786091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b2207-16f7-4197-a761-cc7aeece5946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:01.786710 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b2207-16f7-4197-a761-cc7aeece5946 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:01.786993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b2207-16f7-4197-a761-cc7aeece5946 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:01.787896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b2207-16f7-4197-a761-cc7aeece5946 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:01.800840 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:01.874733 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b2207-16f7-4197-a761-cc7aeece5946 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:01.876175 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b2207-16f7-4197-a761-cc7aeece5946 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:01.891971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d46eb8-04eb-4b53-a00b-647124d7b16a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:01.937818 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2399620-b4ad-4b1c-9200-04f4ee5c2c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:01.938455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2399620-b4ad-4b1c-9200-04f4ee5c2c57 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:01.938727 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2399620-b4ad-4b1c-9200-04f4ee5c2c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:01.938999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2399620-b4ad-4b1c-9200-04f4ee5c2c57 None admin] PATH_INFO: `/v3/groups/3165bc6806024028b3f922fef1cdc82d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:01.940698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:01.941031 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2399620-b4ad-4b1c-9200-04f4ee5c2c57 None admin] RBAC: Authorizing `identity:get_group(group_id=3165bc6806024028b3f922fef1cdc82d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:01.942661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2399620-b4ad-4b1c-9200-04f4ee5c2c57 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:01.944586 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 164/326] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Oct 14 02:12:01 2022] GET /identity/v3/groups/3165bc6806024028b3f922fef1cdc82d => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:01.945496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2399620-b4ad-4b1c-9200-04f4ee5c2c57 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:02.006297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 163/327] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:12:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:12:02.006297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b2207-16f7-4197-a761-cc7aeece5946 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:02.060110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:02.069063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2399620-b4ad-4b1c-9200-04f4ee5c2c57 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:02.101545 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772e362-f1ed-4086-9351-ed007b9a8e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:02.102156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772e362-f1ed-4086-9351-ed007b9a8e9d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:02.102434 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772e362-f1ed-4086-9351-ed007b9a8e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:02.102707 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772e362-f1ed-4086-9351-ed007b9a8e9d None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:02.103408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772e362-f1ed-4086-9351-ed007b9a8e9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:02.103712 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772e362-f1ed-4086-9351-ed007b9a8e9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:02.105074 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772e362-f1ed-4086-9351-ed007b9a8e9d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:02.113852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 165/328] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:12:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:02.114461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772e362-f1ed-4086-9351-ed007b9a8e9d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:02.120492 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b2207-16f7-4197-a761-cc7aeece5946 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:02.155604 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066917f-a0ae-4abd-9a60-210666c7c226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:02.156141 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066917f-a0ae-4abd-9a60-210666c7c226 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:02.156422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066917f-a0ae-4abd-9a60-210666c7c226 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:02.156693 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066917f-a0ae-4abd-9a60-210666c7c226 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:02.157417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066917f-a0ae-4abd-9a60-210666c7c226 None admin] RBAC: Adding query filter params (group.id=3165bc6806024028b3f922fef1cdc82d, role.id=e460c7cf7b5b4c35a5a0e326f69a9fe3, scope.project.id=f94fce41714c4f3099c1ff9eb62caf49) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:02.158293 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066917f-a0ae-4abd-9a60-210666c7c226 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:02.160120 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066917f-a0ae-4abd-9a60-210666c7c226 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:02.165795 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 164/329] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Oct 14 02:12:02 2022] GET /identity/v3/role_assignments?group.id=3165bc6806024028b3f922fef1cdc82d&scope.project.id=f94fce41714c4f3099c1ff9eb62caf49&role.id=e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:02.165795 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066917f-a0ae-4abd-9a60-210666c7c226 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:02.199984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:02.226038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772e362-f1ed-4086-9351-ed007b9a8e9d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:02.265733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7ac51-8919-4db0-be98-40362a56b9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:02.266384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7ac51-8919-4db0-be98-40362a56b9bb None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:02.266664 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7ac51-8919-4db0-be98-40362a56b9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:02.266936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7ac51-8919-4db0-be98-40362a56b9bb None admin] PATH_INFO: `/v3/projects/f94fce41714c4f3099c1ff9eb62caf49` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:02.269007 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7ac51-8919-4db0-be98-40362a56b9bb None admin] RBAC: Authorizing `identity:get_project(project_id=f94fce41714c4f3099c1ff9eb62caf49)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:02.270854 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7ac51-8919-4db0-be98-40362a56b9bb None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:02.277841 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 166/330] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:12:02 2022] GET /identity/v3/projects/f94fce41714c4f3099c1ff9eb62caf49 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:02.278952 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7ac51-8919-4db0-be98-40362a56b9bb None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:02.302488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066917f-a0ae-4abd-9a60-210666c7c226 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:02.309215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:02.342938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa20feb-f55b-455a-be01-dd8268557c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:02.344057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa20feb-f55b-455a-be01-dd8268557c45 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:02.344351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa20feb-f55b-455a-be01-dd8268557c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:02.344624 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa20feb-f55b-455a-be01-dd8268557c45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:02.401266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7ac51-8919-4db0-be98-40362a56b9bb None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:02.421908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa20feb-f55b-455a-be01-dd8268557c45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:02.423960 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa20feb-f55b-455a-be01-dd8268557c45 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:02.442118 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e21b30-b5e7-46c8-aaa1-f82faae40d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:02.442765 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e21b30-b5e7-46c8-aaa1-f82faae40d9e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:02.442892 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:02.443047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e21b30-b5e7-46c8-aaa1-f82faae40d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:02.443367 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e21b30-b5e7-46c8-aaa1-f82faae40d9e None admin] PATH_INFO: `/v3/groups/3165bc6806024028b3f922fef1cdc82d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:02.445566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e21b30-b5e7-46c8-aaa1-f82faae40d9e None admin] RBAC: Authorizing `identity:get_group(group_id=3165bc6806024028b3f922fef1cdc82d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:02.447140 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e21b30-b5e7-46c8-aaa1-f82faae40d9e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:02.448861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 167/331] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Oct 14 02:12:02 2022] GET /identity/v3/groups/3165bc6806024028b3f922fef1cdc82d => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:02.454928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e21b30-b5e7-46c8-aaa1-f82faae40d9e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:02.545960 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 165/332] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:12:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:12:02.547404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa20feb-f55b-455a-be01-dd8268557c45 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:02.563651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:02.572942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e21b30-b5e7-46c8-aaa1-f82faae40d9e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:02.606846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558ad78-a87e-4e1f-9e12-a4b7f54e5530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:02.607473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558ad78-a87e-4e1f-9e12-a4b7f54e5530 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:02.607752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558ad78-a87e-4e1f-9e12-a4b7f54e5530 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:02.608024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558ad78-a87e-4e1f-9e12-a4b7f54e5530 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:02.608749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558ad78-a87e-4e1f-9e12-a4b7f54e5530 None admin] RBAC: Adding query filter params (group.id=3165bc6806024028b3f922fef1cdc82d, role.id=a3287688078248ecb19a6d9c85839631, scope.project.id=f94fce41714c4f3099c1ff9eb62caf49) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:02.609051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558ad78-a87e-4e1f-9e12-a4b7f54e5530 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:02.610480 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558ad78-a87e-4e1f-9e12-a4b7f54e5530 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:02.616345 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 168/333] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Oct 14 02:12:02 2022] GET /identity/v3/role_assignments?group.id=3165bc6806024028b3f922fef1cdc82d&scope.project.id=f94fce41714c4f3099c1ff9eb62caf49&role.id=a3287688078248ecb19a6d9c85839631 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:02.649616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa20feb-f55b-455a-be01-dd8268557c45 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:02.669229 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:02.687922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc51d3-50a2-43a8-b7f8-5cb1a2049074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:02.688491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc51d3-50a2-43a8-b7f8-5cb1a2049074 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:02.688772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc51d3-50a2-43a8-b7f8-5cb1a2049074 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:02.689211 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc51d3-50a2-43a8-b7f8-5cb1a2049074 None admin] PATH_INFO: `/v3/projects/c1e7392d098a4e219827c4ae7f8117e2` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:02.690546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc51d3-50a2-43a8-b7f8-5cb1a2049074 None admin] RBAC: Authorizing `identity:get_project(project_id=c1e7392d098a4e219827c4ae7f8117e2)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:02.692075 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc51d3-50a2-43a8-b7f8-5cb1a2049074 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:02.693780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 166/334] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:12:02 2022] GET /identity/v3/projects/c1e7392d098a4e219827c4ae7f8117e2 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:02.699714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558ad78-a87e-4e1f-9e12-a4b7f54e5530 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:02.808167 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558ad78-a87e-4e1f-9e12-a4b7f54e5530 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:02.842037 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2fc30-86b5-447f-ba23-fc724dcf2132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:02.842667 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2fc30-86b5-447f-ba23-fc724dcf2132 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:02.842942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2fc30-86b5-447f-ba23-fc724dcf2132 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:02.843252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2fc30-86b5-447f-ba23-fc724dcf2132 None admin] PATH_INFO: `/v3/groups/78c813fdca9a48db8990e38f688d2856` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:02.844363 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2fc30-86b5-447f-ba23-fc724dcf2132 None admin] RBAC: Authorizing `identity:get_group(group_id=78c813fdca9a48db8990e38f688d2856)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:02.846425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2fc30-86b5-447f-ba23-fc724dcf2132 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:02.848038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 169/335] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Oct 14 02:12:02 2022] GET /identity/v3/groups/78c813fdca9a48db8990e38f688d2856 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:02.853925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc51d3-50a2-43a8-b7f8-5cb1a2049074 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:02.959571 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc51d3-50a2-43a8-b7f8-5cb1a2049074 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:02.993079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef68a0f-20c0-4919-b103-58f5df6f1fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:02.993689 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef68a0f-20c0-4919-b103-58f5df6f1fdd None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:02.993970 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef68a0f-20c0-4919-b103-58f5df6f1fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:02.994238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef68a0f-20c0-4919-b103-58f5df6f1fdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:02.994957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef68a0f-20c0-4919-b103-58f5df6f1fdd None admin] RBAC: Adding query filter params (group.id=78c813fdca9a48db8990e38f688d2856, role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b, scope.project.id=c1e7392d098a4e219827c4ae7f8117e2) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:02.995303 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef68a0f-20c0-4919-b103-58f5df6f1fdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:02.996737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef68a0f-20c0-4919-b103-58f5df6f1fdd None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:03.003535 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 167/336] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Oct 14 02:12:02 2022] GET /identity/v3/role_assignments?group.id=78c813fdca9a48db8990e38f688d2856&scope.project.id=c1e7392d098a4e219827c4ae7f8117e2&role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:05.565833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe117e-596a-439d-9d29-ab3887c7f60d None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:05.566944 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe117e-596a-439d-9d29-ab3887c7f60d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:05.567462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe117e-596a-439d-9d29-ab3887c7f60d None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:05.569379 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 170/337] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:05.574546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074da4df-7602-42d5-96aa-f1a6b1bc3398 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:05.575453 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074da4df-7602-42d5-96aa-f1a6b1bc3398 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:05.575938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074da4df-7602-42d5-96aa-f1a6b1bc3398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:05.604315 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-074da4df-7602-42d5-96aa-f1a6b1bc3398 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:05.605925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-074da4df-7602-42d5-96aa-f1a6b1bc3398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:05.669017 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:05.695546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 168/338] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:05.723404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65ed3-aa15-4fb4-84dc-47a079c8847e None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:05.723710 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65ed3-aa15-4fb4-84dc-47a079c8847e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:05.723965 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65ed3-aa15-4fb4-84dc-47a079c8847e None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:05.725323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 171/339] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:05.731864 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0fb3f-f83e-46fe-a3ce-c0db7fc5745d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:05.732148 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0fb3f-f83e-46fe-a3ce-c0db7fc5745d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:05.732422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0fb3f-f83e-46fe-a3ce-c0db7fc5745d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:05.768537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-8ad0fb3f-f83e-46fe-a3ce-c0db7fc5745d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:05.769778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8ad0fb3f-f83e-46fe-a3ce-c0db7fc5745d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:05.863870 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 169/340] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:06.124161 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c65ed3-aa15-4fb4-84dc-47a079c8847e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:06.156255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:06.242106 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdef031-f9e5-459d-8cc2-7bc099965436 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:06.242451 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdef031-f9e5-459d-8cc2-7bc099965436 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:06.242644 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdef031-f9e5-459d-8cc2-7bc099965436 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:06.244030 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 170/341] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:06.253029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed106075-bae7-44f6-9629-761465e85967 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:06.253335 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed106075-bae7-44f6-9629-761465e85967 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:06.253578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed106075-bae7-44f6-9629-761465e85967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:06.264451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c65ed3-aa15-4fb4-84dc-47a079c8847e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:06.292059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ed106075-bae7-44f6-9629-761465e85967 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:06.293252 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ed106075-bae7-44f6-9629-761465e85967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:06.306526 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d439444-9990-49a3-8b81-e0d53b481721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:06.309151 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d439444-9990-49a3-8b81-e0d53b481721 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:06.309679 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d439444-9990-49a3-8b81-e0d53b481721 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:06.310360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d439444-9990-49a3-8b81-e0d53b481721 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:06.332252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d439444-9990-49a3-8b81-e0d53b481721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:06.334044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d439444-9990-49a3-8b81-e0d53b481721 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-4d439444-9990-49a3-8b81-e0d53b481721 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:06.359535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:06.360606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:06.361833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:06.361833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:06.361833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:06.361833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 172/342] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:06.362073 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d439444-9990-49a3-8b81-e0d53b481721 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:06.400361 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 171/343] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:06.401379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed106075-bae7-44f6-9629-761465e85967 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:06.448799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:06.454082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:06.478952 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d439444-9990-49a3-8b81-e0d53b481721 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:06.519690 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed106075-bae7-44f6-9629-761465e85967 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:06.524927 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6714603-2c35-4ea4-aff2-26db0b2f8127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:06.525572 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6714603-2c35-4ea4-aff2-26db0b2f8127 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:06.525849 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6714603-2c35-4ea4-aff2-26db0b2f8127 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:06.526158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6714603-2c35-4ea4-aff2-26db0b2f8127 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:06.540724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6714603-2c35-4ea4-aff2-26db0b2f8127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:06.542208 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6714603-2c35-4ea4-aff2-26db0b2f8127 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:06.557626 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2679f2c-b1d9-47cb-92b7-b617c5780437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:06.558582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2679f2c-b1d9-47cb-92b7-b617c5780437 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:06.559150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2679f2c-b1d9-47cb-92b7-b617c5780437 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:06.559631 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2679f2c-b1d9-47cb-92b7-b617c5780437 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:06.560492 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2679f2c-b1d9-47cb-92b7-b617c5780437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:06.561041 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2679f2c-b1d9-47cb-92b7-b617c5780437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-d6714603-2c35-4ea4-aff2-26db0b2f8127 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:06.561158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:06.561938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:06.562823 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:06.562823 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:06.562823 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:06.562823 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 173/344] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:06.564510 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077babf-f348-4eaf-b1a6-4a1d31993da7 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:06.564795 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077babf-f348-4eaf-b1a6-4a1d31993da7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:06.565065 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077babf-f348-4eaf-b1a6-4a1d31993da7 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:06.566240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2679f2c-b1d9-47cb-92b7-b617c5780437 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:06.570747 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 174/345] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:06.571582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077babf-f348-4eaf-b1a6-4a1d31993da7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:06.582557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 172/346] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:06.584590 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52290ee6-052d-496d-859f-2bb1d4434950 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:06.585356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52290ee6-052d-496d-859f-2bb1d4434950 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:06.586059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52290ee6-052d-496d-859f-2bb1d4434950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:06.606277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-52290ee6-052d-496d-859f-2bb1d4434950 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:06.607997 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-52290ee6-052d-496d-859f-2bb1d4434950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:06.690585 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077babf-f348-4eaf-b1a6-4a1d31993da7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:06.694853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 173/347] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:06.695526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52290ee6-052d-496d-859f-2bb1d4434950 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:06.725955 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:06.733274 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03728dd-d1e8-4995-bf69-c2dce934a10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:06.734042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:06.735768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03728dd-d1e8-4995-bf69-c2dce934a10d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:06.736272 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03728dd-d1e8-4995-bf69-c2dce934a10d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:06.737031 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03728dd-d1e8-4995-bf69-c2dce934a10d None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:06.738292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03728dd-d1e8-4995-bf69-c2dce934a10d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:06.738743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03728dd-d1e8-4995-bf69-c2dce934a10d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:06.740292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03728dd-d1e8-4995-bf69-c2dce934a10d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:06.755955 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 175/348] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:06.756997 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03728dd-d1e8-4995-bf69-c2dce934a10d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:06.818401 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52290ee6-052d-496d-859f-2bb1d4434950 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:06.860044 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae04c1d-6857-4ded-87e3-0662804ea4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:06.860896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae04c1d-6857-4ded-87e3-0662804ea4fe None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:06.861269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae04c1d-6857-4ded-87e3-0662804ea4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:06.861874 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae04c1d-6857-4ded-87e3-0662804ea4fe None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:06.862667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae04c1d-6857-4ded-87e3-0662804ea4fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:06.864456 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae04c1d-6857-4ded-87e3-0662804ea4fe None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:06.889894 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03728dd-d1e8-4995-bf69-c2dce934a10d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:06.899095 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 174/349] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:12:06 2022] POST /identity/v3/users => generated 246 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:06.899095 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae04c1d-6857-4ded-87e3-0662804ea4fe None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:06.958756 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c24ea-6aea-4756-8d36-5dc0d24ef3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:06.959651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c24ea-6aea-4756-8d36-5dc0d24ef3bf None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:06.960180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c24ea-6aea-4756-8d36-5dc0d24ef3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:06.960581 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c24ea-6aea-4756-8d36-5dc0d24ef3bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:06.983711 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c24ea-6aea-4756-8d36-5dc0d24ef3bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:06.985789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c24ea-6aea-4756-8d36-5dc0d24ef3bf None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-ed9c24ea-6aea-4756-8d36-5dc0d24ef3bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:07.005380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:07.006405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:07.007229 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:07.007229 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:07.007229 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:07.011731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 176/350] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:07.012387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:07.012766 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c24ea-6aea-4756-8d36-5dc0d24ef3bf None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:07.023821 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae04c1d-6857-4ded-87e3-0662804ea4fe None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:07.074682 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:07.081277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908d16d-ebff-480f-bc55-79593c8ed684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:07.082478 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908d16d-ebff-480f-bc55-79593c8ed684 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:07.082755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908d16d-ebff-480f-bc55-79593c8ed684 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:07.083321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908d16d-ebff-480f-bc55-79593c8ed684 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:07.083987 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908d16d-ebff-480f-bc55-79593c8ed684 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:07.086300 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908d16d-ebff-480f-bc55-79593c8ed684 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:07.123618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 175/351] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:12:06 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:07.124640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908d16d-ebff-480f-bc55-79593c8ed684 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:07.166754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c24ea-6aea-4756-8d36-5dc0d24ef3bf None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:07.213614 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d402087-3eba-451c-ab5f-88915fa2460d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:07.214337 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d402087-3eba-451c-ab5f-88915fa2460d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:07.214613 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d402087-3eba-451c-ab5f-88915fa2460d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:07.214881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d402087-3eba-451c-ab5f-88915fa2460d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:07.228585 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d402087-3eba-451c-ab5f-88915fa2460d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:07.230432 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d402087-3eba-451c-ab5f-88915fa2460d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-6d402087-3eba-451c-ab5f-88915fa2460d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:07.252315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:07.253270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:07.254071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:07.254071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:07.254071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:07.254776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 177/352] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:07.256049 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ca5cc-0414-49b6-919d-0917f9dd407b None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:07.256491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ca5cc-0414-49b6-919d-0917f9dd407b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:07.256767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ca5cc-0414-49b6-919d-0917f9dd407b None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:07.258852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 178/353] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:07.268532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ca5cc-0414-49b6-919d-0917f9dd407b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:07.275980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908d16d-ebff-480f-bc55-79593c8ed684 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:07.333096 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533812d4-c611-4ed8-acfa-7ea25c084c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:07.333783 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533812d4-c611-4ed8-acfa-7ea25c084c20 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:07.334066 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533812d4-c611-4ed8-acfa-7ea25c084c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:07.334335 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533812d4-c611-4ed8-acfa-7ea25c084c20 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:07.334984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533812d4-c611-4ed8-acfa-7ea25c084c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:07.335381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533812d4-c611-4ed8-acfa-7ea25c084c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:07.336749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533812d4-c611-4ed8-acfa-7ea25c084c20 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:07.350414 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 176/354] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:07.351686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9705dc3-6bf5-48d0-b782-5452f20593ea None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:07.351973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9705dc3-6bf5-48d0-b782-5452f20593ea None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:07.352244 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9705dc3-6bf5-48d0-b782-5452f20593ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:07.358433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:07.374128 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d9705dc3-6bf5-48d0-b782-5452f20593ea None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:07.375046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d9705dc3-6bf5-48d0-b782-5452f20593ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:07.385350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:07.396754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ca5cc-0414-49b6-919d-0917f9dd407b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:07.433870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56cfe0-e24c-47bd-a569-9bdf34dbf626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:07.434564 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56cfe0-e24c-47bd-a569-9bdf34dbf626 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:07.434846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56cfe0-e24c-47bd-a569-9bdf34dbf626 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:07.435183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56cfe0-e24c-47bd-a569-9bdf34dbf626 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:07.435833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56cfe0-e24c-47bd-a569-9bdf34dbf626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:07.436136 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56cfe0-e24c-47bd-a569-9bdf34dbf626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:07.437628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56cfe0-e24c-47bd-a569-9bdf34dbf626 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:07.457190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 179/355] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:07.462712 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56cfe0-e24c-47bd-a569-9bdf34dbf626 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:07.467215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 177/356] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:07.468488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9705dc3-6bf5-48d0-b782-5452f20593ea None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:07.637375 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9705dc3-6bf5-48d0-b782-5452f20593ea None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:07.685615 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56cfe0-e24c-47bd-a569-9bdf34dbf626 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:07.696749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:07.708528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f60eb-cccf-43a5-a9c1-ef7a00565b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:07.709158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f60eb-cccf-43a5-a9c1-ef7a00565b69 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:07.709436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f60eb-cccf-43a5-a9c1-ef7a00565b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:07.709706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f60eb-cccf-43a5-a9c1-ef7a00565b69 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:07.710354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f60eb-cccf-43a5-a9c1-ef7a00565b69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:07.714435 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f60eb-cccf-43a5-a9c1-ef7a00565b69 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:07.750560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 178/357] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:12:07 2022] POST /identity/v3/users => generated 248 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:07.751530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d8e2e-2f01-4f25-b94a-2735feb21e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:07.752141 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d8e2e-2f01-4f25-b94a-2735feb21e1a None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:07.752413 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d8e2e-2f01-4f25-b94a-2735feb21e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:07.752729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d8e2e-2f01-4f25-b94a-2735feb21e1a None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:07.753429 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d8e2e-2f01-4f25-b94a-2735feb21e1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:07.755011 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d8e2e-2f01-4f25-b94a-2735feb21e1a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:07.761049 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f60eb-cccf-43a5-a9c1-ef7a00565b69 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:07.787056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 180/358] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:12:07 2022] POST /identity/v3/users => generated 249 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:07.933764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f60eb-cccf-43a5-a9c1-ef7a00565b69 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:07.984313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758dcca1-0cbc-44c4-b05a-6ff84aedaccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:07.985175 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758dcca1-0cbc-44c4-b05a-6ff84aedaccd None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:07.985979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758dcca1-0cbc-44c4-b05a-6ff84aedaccd None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:07.986257 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758dcca1-0cbc-44c4-b05a-6ff84aedaccd None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:07.986937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758dcca1-0cbc-44c4-b05a-6ff84aedaccd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:07.988422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758dcca1-0cbc-44c4-b05a-6ff84aedaccd None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:07.997190 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.sql.core [None req-758dcca1-0cbc-44c4-b05a-6ff84aedaccd None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 14 02:12:07.997190 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 14 02:12:07.997190 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [parameters: {'id': '0160b90aefa84309bda9130c5b04163b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 14 02:12:07.997190 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81362) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-758dcca1-0cbc-44c4-b05a-6ff84aedaccd None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result.read() Oct 14 02:12:08.005194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return ref.to_dict() Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:12:08.006053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application yield resource Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise self.value Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application session.commit() Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 02:12:08.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.session.flush() Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application compat.raise_( Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 02:12:08.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:12:08.008695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result.read() Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:12:08.009454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [parameters: {'id': '0160b90aefa84309bda9130c5b04163b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:08.010215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 14 02:12:08.011020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:08.014028 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 179/359] 10.222.0.18 () {60 vars in 1131 bytes} [Fri Oct 14 02:12:07 2022] POST /identity/v3/roles => generated 144 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:08.018353 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d8e2e-2f01-4f25-b94a-2735feb21e1a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:08.033384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:08.148414 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d8e2e-2f01-4f25-b94a-2735feb21e1a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:08.193608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935636ba-5f48-437d-81ee-c951d9989ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:08.194330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935636ba-5f48-437d-81ee-c951d9989ffa None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:08.194611 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935636ba-5f48-437d-81ee-c951d9989ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:08.194886 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935636ba-5f48-437d-81ee-c951d9989ffa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:08.205907 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935636ba-5f48-437d-81ee-c951d9989ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-935636ba-5f48-437d-81ee-c951d9989ffa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:08.211377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:08.212239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:08.212983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:08.212983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:08.212983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:08.212983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:08.212983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:08.212983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:08.212983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:08.212983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:08.212983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:08.212983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:08.212983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:12:08.212983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:08.212983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 181/360] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:12:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:08.219576 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758dcca1-0cbc-44c4-b05a-6ff84aedaccd None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:08.303757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:08.369660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758dcca1-0cbc-44c4-b05a-6ff84aedaccd None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:08.410207 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2e085-2864-4319-85df-b748e0e9474a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:08.411219 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2e085-2864-4319-85df-b748e0e9474a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:08.411703 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2e085-2864-4319-85df-b748e0e9474a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:08.412162 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2e085-2864-4319-85df-b748e0e9474a None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:08.413086 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2e085-2864-4319-85df-b748e0e9474a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:08.413594 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2e085-2864-4319-85df-b748e0e9474a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:08.415295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2e085-2864-4319-85df-b748e0e9474a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:08.424583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 180/361] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:09.481046 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2454413-3ccb-4b71-8e35-3a99423ce182 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:09.481591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2454413-3ccb-4b71-8e35-3a99423ce182 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:09.481591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2454413-3ccb-4b71-8e35-3a99423ce182 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:09.483266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 182/362] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:09.488341 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb9d9c-12dc-4b1e-9df6-ac3ff3611ea9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:09.488727 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb9d9c-12dc-4b1e-9df6-ac3ff3611ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:09.488900 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb9d9c-12dc-4b1e-9df6-ac3ff3611ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:09.511741 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-63cb9d9c-12dc-4b1e-9df6-ac3ff3611ea9 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:09.512943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-63cb9d9c-12dc-4b1e-9df6-ac3ff3611ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:09.604445 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 181/363] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:09.786100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52045cc9-6d78-4b27-b1c3-be16f8b22912 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:09.786428 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52045cc9-6d78-4b27-b1c3-be16f8b22912 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:09.786654 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52045cc9-6d78-4b27-b1c3-be16f8b22912 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:09.788308 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 183/364] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:09.793314 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9543eb-9965-4ec7-b024-e41b40c1c5ca None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:09.793598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9543eb-9965-4ec7-b024-e41b40c1c5ca None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:09.793871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9543eb-9965-4ec7-b024-e41b40c1c5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:09.824027 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-af9543eb-9965-4ec7-b024-e41b40c1c5ca None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:09.825268 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-af9543eb-9965-4ec7-b024-e41b40c1c5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:09.872709 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:09.912376 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 182/365] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:10.052728 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52045cc9-6d78-4b27-b1c3-be16f8b22912 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:10.133035 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:10.191743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52045cc9-6d78-4b27-b1c3-be16f8b22912 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:10.230980 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c143c-d834-431e-a98b-0885a468b875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:10.231644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c143c-d834-431e-a98b-0885a468b875 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:10.231924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c143c-d834-431e-a98b-0885a468b875 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:10.232198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c143c-d834-431e-a98b-0885a468b875 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:10.246624 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c143c-d834-431e-a98b-0885a468b875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a29c143c-d834-431e-a98b-0885a468b875 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:10.252722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:10.253583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:10.254344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:10.254344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:10.254344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:10.254344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:10.254344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:10.254344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:10.254344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:10.254344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:10.254344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:10.254344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:10.254344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:10.254344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:10.254344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 184/366] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:12:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:10.259711 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9543eb-9965-4ec7-b024-e41b40c1c5ca None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:10.271278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c143c-d834-431e-a98b-0885a468b875 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:10.412042 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c143c-d834-431e-a98b-0885a468b875 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:10.440069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:10.448248 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb36ba-5648-4219-9b28-bc440367bca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:10.448923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb36ba-5648-4219-9b28-bc440367bca7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:10.449205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb36ba-5648-4219-9b28-bc440367bca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:10.449551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb36ba-5648-4219-9b28-bc440367bca7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:10.467620 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb36ba-5648-4219-9b28-bc440367bca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-75bb36ba-5648-4219-9b28-bc440367bca7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:10.468081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:10.468924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:10.470246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:10.470246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:10.470246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:10.470246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:10.470246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:10.470246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:10.470246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:10.470246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:10.470246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:10.470246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:10.470246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:10.470246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:10.470246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 185/367] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:12:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:10.476275 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb36ba-5648-4219-9b28-bc440367bca7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:10.478606 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9543eb-9965-4ec7-b024-e41b40c1c5ca None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:10.539848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf77548-ee1b-4537-a231-4c36414b025d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:10.541265 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf77548-ee1b-4537-a231-4c36414b025d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:10.541690 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf77548-ee1b-4537-a231-4c36414b025d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:10.541963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf77548-ee1b-4537-a231-4c36414b025d None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:10.542635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf77548-ee1b-4537-a231-4c36414b025d None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:10.542936 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf77548-ee1b-4537-a231-4c36414b025d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:10.544445 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf77548-ee1b-4537-a231-4c36414b025d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:10.554138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 183/368] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Oct 14 02:12:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:10.560871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa59f53-d38e-4890-8f51-347d054bde95 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:10.560871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa59f53-d38e-4890-8f51-347d054bde95 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:10.560871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa59f53-d38e-4890-8f51-347d054bde95 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:10.560871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 184/369] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:10.561183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa59f53-d38e-4890-8f51-347d054bde95 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:10.600040 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:10.610050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb36ba-5648-4219-9b28-bc440367bca7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:10.647463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a859b-a196-457a-9760-be08988b884e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:10.648070 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a859b-a196-457a-9760-be08988b884e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:10.648342 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a859b-a196-457a-9760-be08988b884e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:10.648611 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a859b-a196-457a-9760-be08988b884e None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:10.649267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a859b-a196-457a-9760-be08988b884e None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:10.649565 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a859b-a196-457a-9760-be08988b884e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:10.650932 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a859b-a196-457a-9760-be08988b884e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:10.659265 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 186/370] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Oct 14 02:12:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:10.660256 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052247c-b4b0-4ed5-b5c7-a1f1ff1b5bd8 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:10.660536 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052247c-b4b0-4ed5-b5c7-a1f1ff1b5bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:10.660802 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052247c-b4b0-4ed5-b5c7-a1f1ff1b5bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:10.674964 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa59f53-d38e-4890-8f51-347d054bde95 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:10.678165 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6052247c-b4b0-4ed5-b5c7-a1f1ff1b5bd8 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:10.679418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6052247c-b4b0-4ed5-b5c7-a1f1ff1b5bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:10.712814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e63900-f098-4e95-ab31-db4a3e7a03a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:10.713766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e63900-f098-4e95-ab31-db4a3e7a03a5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:10.714040 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e63900-f098-4e95-ab31-db4a3e7a03a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:10.714306 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e63900-f098-4e95-ab31-db4a3e7a03a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:10.727155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e63900-f098-4e95-ab31-db4a3e7a03a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:10.728601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e63900-f098-4e95-ab31-db4a3e7a03a5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:10.732037 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-54e63900-f098-4e95-ab31-db4a3e7a03a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:10.743301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:10.744225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:10.745064 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:10.745064 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:10.745064 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:10.747443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 185/371] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:10.749177 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f156b-45af-4d84-bec0-20a989b85a2d None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:10.749604 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f156b-45af-4d84-bec0-20a989b85a2d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:10.749970 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f156b-45af-4d84-bec0-20a989b85a2d None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:10.751959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 186/372] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:10.752520 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f156b-45af-4d84-bec0-20a989b85a2d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:10.755110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 187/373] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:10.762273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052247c-b4b0-4ed5-b5c7-a1f1ff1b5bd8 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:10.857103 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:10.876967 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f156b-45af-4d84-bec0-20a989b85a2d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:10.887284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052247c-b4b0-4ed5-b5c7-a1f1ff1b5bd8 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:10.917270 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fb7d3-d563-4940-8204-fcd48efd3383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:10.918228 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fb7d3-d563-4940-8204-fcd48efd3383 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:10.918694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fb7d3-d563-4940-8204-fcd48efd3383 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:10.919180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fb7d3-d563-4940-8204-fcd48efd3383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:10.931701 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fb7d3-d563-4940-8204-fcd48efd3383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:10.933854 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fb7d3-d563-4940-8204-fcd48efd3383 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:10.935636 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbaf5ac-7b59-4014-9212-1ae2fa31dc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:10.936517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaf5ac-7b59-4014-9212-1ae2fa31dc8c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:10.936796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaf5ac-7b59-4014-9212-1ae2fa31dc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:10.937066 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaf5ac-7b59-4014-9212-1ae2fa31dc8c None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:10.938149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbaf5ac-7b59-4014-9212-1ae2fa31dc8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:10.938544 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbaf5ac-7b59-4014-9212-1ae2fa31dc8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:10.939974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbaf5ac-7b59-4014-9212-1ae2fa31dc8c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-023fb7d3-d563-4940-8204-fcd48efd3383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:10.952796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:10.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:10.955015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:10.955015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:10.955015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:10.961171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 188/375] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:10.961171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a213edc-5f80-4975-82d4-00b79e346d14 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:10.961171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a213edc-5f80-4975-82d4-00b79e346d14 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:10.961171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a213edc-5f80-4975-82d4-00b79e346d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:10.961631 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 187/375] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:10.962663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fb7d3-d563-4940-8204-fcd48efd3383 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:10.982785 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3a213edc-5f80-4975-82d4-00b79e346d14 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:10.984676 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3a213edc-5f80-4975-82d4-00b79e346d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:11.035128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:11.072879 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 189/376] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:11.074390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fb7d3-d563-4940-8204-fcd48efd3383 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:11.076198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a213edc-5f80-4975-82d4-00b79e346d14 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:11.110848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104551b-edcd-4262-8d45-09043527c316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:11.111567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104551b-edcd-4262-8d45-09043527c316 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:11.111846 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104551b-edcd-4262-8d45-09043527c316 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:11.112119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104551b-edcd-4262-8d45-09043527c316 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:11.120434 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104551b-edcd-4262-8d45-09043527c316 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:11.122155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104551b-edcd-4262-8d45-09043527c316 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:11.125363 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-4104551b-edcd-4262-8d45-09043527c316 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:11.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:11.138650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:11.139549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:11.139549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:11.139549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:11.139549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:11.139549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:11.139549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:11.139549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:11.139549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:11.139549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:12:11.139549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:11.139549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 188/377] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Oct 14 02:12:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:11.144631 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104551b-edcd-4262-8d45-09043527c316 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:11.221610 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a213edc-5f80-4975-82d4-00b79e346d14 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:11.255193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104551b-edcd-4262-8d45-09043527c316 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:11.272597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99a6d8-d5dc-43c8-b5ae-435118403d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:11.273831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99a6d8-d5dc-43c8-b5ae-435118403d04 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:11.274307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99a6d8-d5dc-43c8-b5ae-435118403d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:11.274757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99a6d8-d5dc-43c8-b5ae-435118403d04 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:11.275651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99a6d8-d5dc-43c8-b5ae-435118403d04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:11.276650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99a6d8-d5dc-43c8-b5ae-435118403d04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:11.278555 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99a6d8-d5dc-43c8-b5ae-435118403d04 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:11.291837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0bba6-f880-4f83-84fe-065f5bc4e3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:11.292570 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0bba6-f880-4f83-84fe-065f5bc4e3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:11.292937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0bba6-f880-4f83-84fe-065f5bc4e3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:11.293278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0bba6-f880-4f83-84fe-065f5bc4e3f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:11.299612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 190/378] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:11.300371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99a6d8-d5dc-43c8-b5ae-435118403d04 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:11.310721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea0bba6-f880-4f83-84fe-065f5bc4e3f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-1ea0bba6-f880-4f83-84fe-065f5bc4e3f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:11.316774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:11.317637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:11.318412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:11.318412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:11.318412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:11.318412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:11.318412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:11.318412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:11.318412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:11.318412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:11.318412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:11.318412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:11.318412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:11.318412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:11.318412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 189/379] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:12:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:11.319040 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0bba6-f880-4f83-84fe-065f5bc4e3f5 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:11.333998 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:11.380810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:11.403593 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99a6d8-d5dc-43c8-b5ae-435118403d04 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:11.422256 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0bba6-f880-4f83-84fe-065f5bc4e3f5 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:11.437403 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95697b18-2985-4d7e-9163-5da3e1a5bdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:11.438007 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95697b18-2985-4d7e-9163-5da3e1a5bdd8 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:11.438279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95697b18-2985-4d7e-9163-5da3e1a5bdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:11.439079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95697b18-2985-4d7e-9163-5da3e1a5bdd8 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:11.439806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95697b18-2985-4d7e-9163-5da3e1a5bdd8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:11.440105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95697b18-2985-4d7e-9163-5da3e1a5bdd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:11.441523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95697b18-2985-4d7e-9163-5da3e1a5bdd8 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:11.455140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31969b83-ef52-4042-ac1f-033369666995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:11.455775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31969b83-ef52-4042-ac1f-033369666995 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:11.456056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31969b83-ef52-4042-ac1f-033369666995 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:11.456329 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31969b83-ef52-4042-ac1f-033369666995 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:11.464703 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31969b83-ef52-4042-ac1f-033369666995 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:11.466156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31969b83-ef52-4042-ac1f-033369666995 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:11.477242 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 191/380] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Oct 14 02:12:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-31969b83-ef52-4042-ac1f-033369666995 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:11.477612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:11.478419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:11.479326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:11.479326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:11.479326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:11.479326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:11.479326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:11.479326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:11.479326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:11.479326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:11.479326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 14 02:12:11.479326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:11.479922 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d877c-dc2c-4b3c-857e-100ec374123f None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:11.479922 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d877c-dc2c-4b3c-857e-100ec374123f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:11.480036 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 190/381] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:12:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:11.480036 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31969b83-ef52-4042-ac1f-033369666995 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:11.480962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d877c-dc2c-4b3c-857e-100ec374123f None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:11.482565 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 192/382] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:11.483328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d877c-dc2c-4b3c-857e-100ec374123f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:11.572089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:11.581571 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31969b83-ef52-4042-ac1f-033369666995 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:11.585639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d877c-dc2c-4b3c-857e-100ec374123f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:11.601059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:11.615942 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf4b66-dc68-4840-b774-1b08eb91624c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:11.616544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf4b66-dc68-4840-b774-1b08eb91624c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:11.616819 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf4b66-dc68-4840-b774-1b08eb91624c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:11.617086 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf4b66-dc68-4840-b774-1b08eb91624c None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:11.617737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf4b66-dc68-4840-b774-1b08eb91624c None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:11.618035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf4b66-dc68-4840-b774-1b08eb91624c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:11.619426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf4b66-dc68-4840-b774-1b08eb91624c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:11.621049 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84aa48f-72df-4d36-82ef-6903a08cee16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:11.621677 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84aa48f-72df-4d36-82ef-6903a08cee16 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:11.621955 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84aa48f-72df-4d36-82ef-6903a08cee16 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:11.622223 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84aa48f-72df-4d36-82ef-6903a08cee16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:11.628165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 191/383] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Oct 14 02:12:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:11.628836 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf4b66-dc68-4840-b774-1b08eb91624c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:11.636203 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84aa48f-72df-4d36-82ef-6903a08cee16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-c84aa48f-72df-4d36-82ef-6903a08cee16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:11.641026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:11.641878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:11.642881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:11.642881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:11.642881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:11.642881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:11.642881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:11.642881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:11.642881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:11.642881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:11.642881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:11.642881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:11.642881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:11.642881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:11.642881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 193/384] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:12:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:11.643583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84aa48f-72df-4d36-82ef-6903a08cee16 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:11.728177 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf4b66-dc68-4840-b774-1b08eb91624c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:11.744587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84aa48f-72df-4d36-82ef-6903a08cee16 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:11.761944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21640179-8b11-4c1c-beb6-37f6eca4dd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:11.762578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21640179-8b11-4c1c-beb6-37f6eca4dd30 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:11.762856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21640179-8b11-4c1c-beb6-37f6eca4dd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:11.763158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21640179-8b11-4c1c-beb6-37f6eca4dd30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:11.773797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21640179-8b11-4c1c-beb6-37f6eca4dd30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:11.775253 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21640179-8b11-4c1c-beb6-37f6eca4dd30 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:11.779050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7350619-c36b-4abd-b6d5-61820d1ba5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:11.779727 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7350619-c36b-4abd-b6d5-61820d1ba5f5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:11.780008 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7350619-c36b-4abd-b6d5-61820d1ba5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:11.780277 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7350619-c36b-4abd-b6d5-61820d1ba5f5 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:11.781006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7350619-c36b-4abd-b6d5-61820d1ba5f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:11.781308 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7350619-c36b-4abd-b6d5-61820d1ba5f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:11.782905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7350619-c36b-4abd-b6d5-61820d1ba5f5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-21640179-8b11-4c1c-beb6-37f6eca4dd30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:11.789104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:11.789952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:11.790768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:11.790768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:11.790768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:11.791135 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 192/385] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:11.792691 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e7818-ce6d-4a0f-b511-fab4e9cf8d82 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:11.792982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e7818-ce6d-4a0f-b511-fab4e9cf8d82 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:11.793260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e7818-ce6d-4a0f-b511-fab4e9cf8d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:11.800880 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 194/386] 10.222.0.18 () {56 vars in 1137 bytes} [Fri Oct 14 02:12:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:11.801704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7350619-c36b-4abd-b6d5-61820d1ba5f5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:11.811276 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:11.812035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4e9e7818-ce6d-4a0f-b511-fab4e9cf8d82 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:11.813644 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4e9e7818-ce6d-4a0f-b511-fab4e9cf8d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:11.839928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:11.880844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 193/387] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:11.881672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e7818-ce6d-4a0f-b511-fab4e9cf8d82 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:11.915368 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7350619-c36b-4abd-b6d5-61820d1ba5f5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:11.951222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacccf5-59bb-42e8-9600-18cce39fcf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:11.951857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacccf5-59bb-42e8-9600-18cce39fcf2c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:11.952137 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacccf5-59bb-42e8-9600-18cce39fcf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:11.952412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacccf5-59bb-42e8-9600-18cce39fcf2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:11.962804 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cacccf5-59bb-42e8-9600-18cce39fcf2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:11.964314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cacccf5-59bb-42e8-9600-18cce39fcf2c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-5cacccf5-59bb-42e8-9600-18cce39fcf2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:11.978901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:11.979923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:11.980741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:11.980741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:11.980741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:11.980741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 195/388] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:11.980741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacccf5-59bb-42e8-9600-18cce39fcf2c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:11.991183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e7818-ce6d-4a0f-b511-fab4e9cf8d82 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:12.028971 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18d9b-d49f-429e-90b9-010dbd9f1a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:12.029613 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18d9b-d49f-429e-90b9-010dbd9f1a99 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:12.029897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18d9b-d49f-429e-90b9-010dbd9f1a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:12.030184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18d9b-d49f-429e-90b9-010dbd9f1a99 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:12.030834 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd18d9b-d49f-429e-90b9-010dbd9f1a99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:12.031167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd18d9b-d49f-429e-90b9-010dbd9f1a99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:12.033138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd18d9b-d49f-429e-90b9-010dbd9f1a99 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:12.041825 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 194/389] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Oct 14 02:12:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:12.042544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18d9b-d49f-429e-90b9-010dbd9f1a99 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:12.045309 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:12.073956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:12.092977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacccf5-59bb-42e8-9600-18cce39fcf2c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:12.128861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58ae32-842f-4d7c-aa95-6fd3cb1b2015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:12.129486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58ae32-842f-4d7c-aa95-6fd3cb1b2015 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:12.129756 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58ae32-842f-4d7c-aa95-6fd3cb1b2015 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:12.130036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58ae32-842f-4d7c-aa95-6fd3cb1b2015 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:12.130664 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58ae32-842f-4d7c-aa95-6fd3cb1b2015 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:12.130972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58ae32-842f-4d7c-aa95-6fd3cb1b2015 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:12.132385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58ae32-842f-4d7c-aa95-6fd3cb1b2015 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:12.143829 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 196/390] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:12.144631 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58ae32-842f-4d7c-aa95-6fd3cb1b2015 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:12.154535 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18d9b-d49f-429e-90b9-010dbd9f1a99 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:12.189383 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2fe76-7257-45e4-b03a-b2b8727a8404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:12.190009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2fe76-7257-45e4-b03a-b2b8727a8404 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:12.190285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2fe76-7257-45e4-b03a-b2b8727a8404 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:12.190554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2fe76-7257-45e4-b03a-b2b8727a8404 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:12.201325 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2fe76-7257-45e4-b03a-b2b8727a8404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:12.202763 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2fe76-7257-45e4-b03a-b2b8727a8404 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-e3e2fe76-7257-45e4-b03a-b2b8727a8404 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:12.216599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:12.217633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:12.218472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:12.218472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:12.218472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:12.218472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 195/391] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:12.218472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2fe76-7257-45e4-b03a-b2b8727a8404 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:12.245353 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58ae32-842f-4d7c-aa95-6fd3cb1b2015 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:12.262328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:12.280183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc77541-cdf3-4c5b-82cc-a488c6b8f642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:12.280790 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77541-cdf3-4c5b-82cc-a488c6b8f642 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:12.281069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77541-cdf3-4c5b-82cc-a488c6b8f642 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:12.281344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77541-cdf3-4c5b-82cc-a488c6b8f642 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:12.281979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc77541-cdf3-4c5b-82cc-a488c6b8f642 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:12.282849 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc77541-cdf3-4c5b-82cc-a488c6b8f642 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:12.284249 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc77541-cdf3-4c5b-82cc-a488c6b8f642 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:12.295496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 197/392] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:12.296283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc77541-cdf3-4c5b-82cc-a488c6b8f642 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:12.300918 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:12.323550 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2fe76-7257-45e4-b03a-b2b8727a8404 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:12.356285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b5138-463e-43d9-8600-c5893d49bf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:12.356910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b5138-463e-43d9-8600-c5893d49bf48 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:12.357187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b5138-463e-43d9-8600-c5893d49bf48 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:12.357457 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b5138-463e-43d9-8600-c5893d49bf48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:12.368167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b5138-463e-43d9-8600-c5893d49bf48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:12.369601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b5138-463e-43d9-8600-c5893d49bf48 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-dd9b5138-463e-43d9-8600-c5893d49bf48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:12.383828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:12.384988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:12.385791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:12.385791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:12.385791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:12.385791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 196/393] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:12.385791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b5138-463e-43d9-8600-c5893d49bf48 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:12.398983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc77541-cdf3-4c5b-82cc-a488c6b8f642 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:12.434957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a3636-853e-48cb-bd61-6a609cf5b711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:12.435613 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a3636-853e-48cb-bd61-6a609cf5b711 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:12.435888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a3636-853e-48cb-bd61-6a609cf5b711 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:12.436164 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a3636-853e-48cb-bd61-6a609cf5b711 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:12.443627 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a3636-853e-48cb-bd61-6a609cf5b711 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:12.445069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a3636-853e-48cb-bd61-6a609cf5b711 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-3b6a3636-853e-48cb-bd61-6a609cf5b711 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:12.456647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:12.457479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:12.458293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:12.458293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:12.458293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:12.458293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:12.458293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:12.458293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:12.458293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:12.458293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:12.458293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 198/394] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:12.458752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a3636-853e-48cb-bd61-6a609cf5b711 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:12.493388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b5138-463e-43d9-8600-c5893d49bf48 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:12.498177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:12.522008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:12.527997 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb53574-0d72-49dc-b4aa-b4cb1c2ccf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:12.528636 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb53574-0d72-49dc-b4aa-b4cb1c2ccf1d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:12.528914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb53574-0d72-49dc-b4aa-b4cb1c2ccf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:12.529189 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb53574-0d72-49dc-b4aa-b4cb1c2ccf1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:12.539762 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb53574-0d72-49dc-b4aa-b4cb1c2ccf1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:12.541299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb53574-0d72-49dc-b4aa-b4cb1c2ccf1d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-3eb53574-0d72-49dc-b4aa-b4cb1c2ccf1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:12.555546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:12.556598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:12.557602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:12.557602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:12.557602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:12.557602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 197/395] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:12.557602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb53574-0d72-49dc-b4aa-b4cb1c2ccf1d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:12.560819 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a3636-853e-48cb-bd61-6a609cf5b711 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:12.593986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e79dc-afdc-4dd3-b018-a19a41a1477b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:12.594621 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e79dc-afdc-4dd3-b018-a19a41a1477b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:12.594905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e79dc-afdc-4dd3-b018-a19a41a1477b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:12.595214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e79dc-afdc-4dd3-b018-a19a41a1477b None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:12.595852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e79dc-afdc-4dd3-b018-a19a41a1477b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:12.596155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e79dc-afdc-4dd3-b018-a19a41a1477b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:12.597513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e79dc-afdc-4dd3-b018-a19a41a1477b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:12.608386 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 199/396] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:12.609166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e79dc-afdc-4dd3-b018-a19a41a1477b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:12.656571 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb53574-0d72-49dc-b4aa-b4cb1c2ccf1d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:12.690696 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bc15d-02d4-4091-b491-fce20e1ed95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:12.691351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bc15d-02d4-4091-b491-fce20e1ed95c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:12.691627 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bc15d-02d4-4091-b491-fce20e1ed95c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:12.691899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bc15d-02d4-4091-b491-fce20e1ed95c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:12.699169 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457bc15d-02d4-4091-b491-fce20e1ed95c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:12.700591 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457bc15d-02d4-4091-b491-fce20e1ed95c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:12.703661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:12.707617 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e79dc-afdc-4dd3-b018-a19a41a1477b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-457bc15d-02d4-4091-b491-fce20e1ed95c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:12.712193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:12.713222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:12.714131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:12.714131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:12.714131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:12.714131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:12.714131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:12.714131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:12.714131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:12.714131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:12.714131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 14 02:12:12.714131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:12.714131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 198/397] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:12:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:12.714686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bc15d-02d4-4091-b491-fce20e1ed95c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:12.744304 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b79b53-fc4f-428a-98e9-a0d7601299a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:12.744913 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b79b53-fc4f-428a-98e9-a0d7601299a7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:12.745193 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b79b53-fc4f-428a-98e9-a0d7601299a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:12.745469 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b79b53-fc4f-428a-98e9-a0d7601299a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:12.746109 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b79b53-fc4f-428a-98e9-a0d7601299a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:12.746410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b79b53-fc4f-428a-98e9-a0d7601299a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:12.748066 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b79b53-fc4f-428a-98e9-a0d7601299a7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:12.758812 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:12.761602 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 200/398] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:12.762342 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b79b53-fc4f-428a-98e9-a0d7601299a7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:12.817011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bc15d-02d4-4091-b491-fce20e1ed95c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:12.852284 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293a155-e630-42e6-9e96-19285651dc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:12.852898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a155-e630-42e6-9e96-19285651dc77 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:12.853198 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a155-e630-42e6-9e96-19285651dc77 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:12.853476 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a155-e630-42e6-9e96-19285651dc77 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:12.854162 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293a155-e630-42e6-9e96-19285651dc77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:12.854467 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293a155-e630-42e6-9e96-19285651dc77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:12.855939 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293a155-e630-42e6-9e96-19285651dc77 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:12.859933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b79b53-fc4f-428a-98e9-a0d7601299a7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:12.873629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 199/399] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:12.874428 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293a155-e630-42e6-9e96-19285651dc77 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:12.895124 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51fa2d-6471-42a3-a92f-a2db9c335757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:12.895727 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51fa2d-6471-42a3-a92f-a2db9c335757 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:12.896000 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51fa2d-6471-42a3-a92f-a2db9c335757 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:12.896269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51fa2d-6471-42a3-a92f-a2db9c335757 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:12.896899 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51fa2d-6471-42a3-a92f-a2db9c335757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:12.897198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51fa2d-6471-42a3-a92f-a2db9c335757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:12.898552 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51fa2d-6471-42a3-a92f-a2db9c335757 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:12.910024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 201/400] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:12.910821 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51fa2d-6471-42a3-a92f-a2db9c335757 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:12.929531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:12.979622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293a155-e630-42e6-9e96-19285651dc77 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:12.981122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:13.008556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51fa2d-6471-42a3-a92f-a2db9c335757 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:13.014099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406a90b-6fe5-4a6e-9114-1c2b720960d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:13.014671 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406a90b-6fe5-4a6e-9114-1c2b720960d3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:13.014949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406a90b-6fe5-4a6e-9114-1c2b720960d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:13.015252 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406a90b-6fe5-4a6e-9114-1c2b720960d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:13.022370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406a90b-6fe5-4a6e-9114-1c2b720960d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:13.023853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406a90b-6fe5-4a6e-9114-1c2b720960d3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-b406a90b-6fe5-4a6e-9114-1c2b720960d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:13.035794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:13.036780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:13.038022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:13.038022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:13.038022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:13.038022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:13.038022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:13.038022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:13.038022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:13.038022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:13.038022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 200/401] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:13.038022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406a90b-6fe5-4a6e-9114-1c2b720960d3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:13.046373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d9fd6-9c02-48f4-af9f-bd90a4380715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:13.046979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d9fd6-9c02-48f4-af9f-bd90a4380715 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:13.047287 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d9fd6-9c02-48f4-af9f-bd90a4380715 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:13.047589 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d9fd6-9c02-48f4-af9f-bd90a4380715 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:13.048624 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d9fd6-9c02-48f4-af9f-bd90a4380715 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:13.048933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d9fd6-9c02-48f4-af9f-bd90a4380715 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:13.050357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d9fd6-9c02-48f4-af9f-bd90a4380715 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:13.067490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 202/402] 10.222.0.18 () {56 vars in 1139 bytes} [Fri Oct 14 02:12:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:13.068094 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d9fd6-9c02-48f4-af9f-bd90a4380715 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:13.140221 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406a90b-6fe5-4a6e-9114-1c2b720960d3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:13.147354 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:13.170309 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d9fd6-9c02-48f4-af9f-bd90a4380715 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:13.173714 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53654015-b962-4cb1-8eae-565052e46797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:13.174337 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53654015-b962-4cb1-8eae-565052e46797 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:13.174613 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53654015-b962-4cb1-8eae-565052e46797 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:13.174880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53654015-b962-4cb1-8eae-565052e46797 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:13.182388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53654015-b962-4cb1-8eae-565052e46797 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:13.183855 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53654015-b962-4cb1-8eae-565052e46797 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-53654015-b962-4cb1-8eae-565052e46797 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:13.194881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:13.195998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:13.197147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:13.197147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:13.197147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:13.197147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:13.197147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:13.197147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:13.197147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:13.197147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:13.197147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 02:12:13.197147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:13.197147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 201/403] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:12:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:13.197147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53654015-b962-4cb1-8eae-565052e46797 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:13.202857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cbdff-988f-4f97-9e5e-80cab6dfc12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:13.203550 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cbdff-988f-4f97-9e5e-80cab6dfc12a None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:13.204358 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cbdff-988f-4f97-9e5e-80cab6dfc12a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:13.204630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cbdff-988f-4f97-9e5e-80cab6dfc12a None admin] PATH_INFO: `/v3/projects/f28d337589024cc88184355bd0ee8572/users/de13be55b4034477baf4c029c1a79757/roles/8adf2afafb0446a6992e9aeb380f645d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:13.218829 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cbdff-988f-4f97-9e5e-80cab6dfc12a None admin] RBAC: Authorizing `identity:create_grant(project_id=f28d337589024cc88184355bd0ee8572, user_id=de13be55b4034477baf4c029c1a79757, role_id=8adf2afafb0446a6992e9aeb380f645d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:13.220414 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cbdff-988f-4f97-9e5e-80cab6dfc12a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:13.225082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:13.232905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 203/404] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:12:13 2022] PUT /identity/v3/projects/f28d337589024cc88184355bd0ee8572/users/de13be55b4034477baf4c029c1a79757/roles/8adf2afafb0446a6992e9aeb380f645d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:12:13.247255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cbdff-988f-4f97-9e5e-80cab6dfc12a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:13.317939 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53654015-b962-4cb1-8eae-565052e46797 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:13.340219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cbdff-988f-4f97-9e5e-80cab6dfc12a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:13.356093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a907da9-659e-4020-a7af-966e98d884c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:13.356726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a907da9-659e-4020-a7af-966e98d884c6 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:13.357011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a907da9-659e-4020-a7af-966e98d884c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:13.357287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a907da9-659e-4020-a7af-966e98d884c6 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:13.357945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a907da9-659e-4020-a7af-966e98d884c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:13.359490 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a907da9-659e-4020-a7af-966e98d884c6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:13.372023 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:13.382871 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4de62-d57f-4573-b320-555cfac0e0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:13.383532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4de62-d57f-4573-b320-555cfac0e0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:13.383814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4de62-d57f-4573-b320-555cfac0e0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:13.384090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4de62-d57f-4573-b320-555cfac0e0e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:13.384794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4de62-d57f-4573-b320-555cfac0e0e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:13.385099 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4de62-d57f-4573-b320-555cfac0e0e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:13.386560 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4de62-d57f-4573-b320-555cfac0e0e7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:13.387473 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 202/405] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:12:13 2022] POST /identity/v3/users => generated 247 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:13.388629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a907da9-659e-4020-a7af-966e98d884c6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:13.434502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 204/406] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:13.435716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4de62-d57f-4573-b320-555cfac0e0e7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:13.522228 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:13.557721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a907da9-659e-4020-a7af-966e98d884c6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:13.577643 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4de62-d57f-4573-b320-555cfac0e0e7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:13.606273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb7ee9-82b8-4659-b8df-9003dfad116a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:13.606920 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb7ee9-82b8-4659-b8df-9003dfad116a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:13.607274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb7ee9-82b8-4659-b8df-9003dfad116a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:13.607744 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb7ee9-82b8-4659-b8df-9003dfad116a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:13.621165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb7ee9-82b8-4659-b8df-9003dfad116a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:13.622889 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb7ee9-82b8-4659-b8df-9003dfad116a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:13.624534 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b5a4d-0780-4237-be47-cd8018c7370e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:13.625144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b5a4d-0780-4237-be47-cd8018c7370e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:13.625418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b5a4d-0780-4237-be47-cd8018c7370e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:13.625720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b5a4d-0780-4237-be47-cd8018c7370e None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:13.626531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b5a4d-0780-4237-be47-cd8018c7370e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:13.626838 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b5a4d-0780-4237-be47-cd8018c7370e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:13.628405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b5a4d-0780-4237-be47-cd8018c7370e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-3afb7ee9-82b8-4659-b8df-9003dfad116a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:13.640687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:13.641815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:13.642853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:13.642853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:13.642853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:13.642853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 203/407] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:13.643161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb7ee9-82b8-4659-b8df-9003dfad116a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:13.653204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 205/408] 10.222.0.18 () {56 vars in 1137 bytes} [Fri Oct 14 02:12:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:13.654333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b5a4d-0780-4237-be47-cd8018c7370e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:13.712349 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:13.771865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b5a4d-0780-4237-be47-cd8018c7370e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:13.783724 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb7ee9-82b8-4659-b8df-9003dfad116a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:13.810788 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828ac11-eab2-43ba-a38c-12cebcb44f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:13.811451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828ac11-eab2-43ba-a38c-12cebcb44f39 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:13.811739 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828ac11-eab2-43ba-a38c-12cebcb44f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:13.812016 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828ac11-eab2-43ba-a38c-12cebcb44f39 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:13.812662 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828ac11-eab2-43ba-a38c-12cebcb44f39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:13.812969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828ac11-eab2-43ba-a38c-12cebcb44f39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:13.814724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828ac11-eab2-43ba-a38c-12cebcb44f39 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:13.821202 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:13.828334 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc84ca5-5302-4dfd-ae8b-f1ddc7a97b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:13.829858 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc84ca5-5302-4dfd-ae8b-f1ddc7a97b8c None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:13.830407 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc84ca5-5302-4dfd-ae8b-f1ddc7a97b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:13.830826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc84ca5-5302-4dfd-ae8b-f1ddc7a97b8c None admin] PATH_INFO: `/v3/projects/f28d337589024cc88184355bd0ee8572/users/8c05a1b303c2418fb2941b8ef56f8972/roles/8adf2afafb0446a6992e9aeb380f645d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:13.831402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 206/409] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:13.832314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828ac11-eab2-43ba-a38c-12cebcb44f39 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:13.850142 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc84ca5-5302-4dfd-ae8b-f1ddc7a97b8c None admin] RBAC: Authorizing `identity:create_grant(project_id=f28d337589024cc88184355bd0ee8572, user_id=8c05a1b303c2418fb2941b8ef56f8972, role_id=8adf2afafb0446a6992e9aeb380f645d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:13.852155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc84ca5-5302-4dfd-ae8b-f1ddc7a97b8c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:13.870304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 204/410] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:12:13 2022] PUT /identity/v3/projects/f28d337589024cc88184355bd0ee8572/users/8c05a1b303c2418fb2941b8ef56f8972/roles/8adf2afafb0446a6992e9aeb380f645d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:12:13.872385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc84ca5-5302-4dfd-ae8b-f1ddc7a97b8c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:13.946958 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828ac11-eab2-43ba-a38c-12cebcb44f39 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:13.950393 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:13.987209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65651a1-e10c-425b-98d5-d850771d3c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:13.988444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65651a1-e10c-425b-98d5-d850771d3c5d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:13.988444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65651a1-e10c-425b-98d5-d850771d3c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:13.988632 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65651a1-e10c-425b-98d5-d850771d3c5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:14.000875 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65651a1-e10c-425b-98d5-d850771d3c5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:14.002642 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65651a1-e10c-425b-98d5-d850771d3c5d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:14.018564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc84ca5-5302-4dfd-ae8b-f1ddc7a97b8c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-b65651a1-e10c-425b-98d5-d850771d3c5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:14.037402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:14.038519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:14.039523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:14.039523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:14.039523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:14.048121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 207/411] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:14.049641 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65651a1-e10c-425b-98d5-d850771d3c5d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:14.062316 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7f1cb-54fc-4e5a-9944-2b9ea8e32ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:14.063321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7f1cb-54fc-4e5a-9944-2b9ea8e32ecd None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:14.063770 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7f1cb-54fc-4e5a-9944-2b9ea8e32ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:14.064870 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7f1cb-54fc-4e5a-9944-2b9ea8e32ecd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:14.073136 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7f1cb-54fc-4e5a-9944-2b9ea8e32ecd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:14.074856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7f1cb-54fc-4e5a-9944-2b9ea8e32ecd None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-a2f7f1cb-54fc-4e5a-9944-2b9ea8e32ecd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:14.090264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:14.091327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:14.092203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:14.092203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:14.092203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:14.092203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:14.092203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:14.092203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:14.092203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:14.092203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:14.094301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 205/412] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:14.103121 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7f1cb-54fc-4e5a-9944-2b9ea8e32ecd None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:14.185118 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:14.229380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65651a1-e10c-425b-98d5-d850771d3c5d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:14.263443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7f1cb-54fc-4e5a-9944-2b9ea8e32ecd None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:14.270532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2844034-1e32-4a10-8a5e-6bfd27c1b6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:14.271467 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2844034-1e32-4a10-8a5e-6bfd27c1b6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:14.271918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2844034-1e32-4a10-8a5e-6bfd27c1b6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:14.272325 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2844034-1e32-4a10-8a5e-6bfd27c1b6f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:14.273201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2844034-1e32-4a10-8a5e-6bfd27c1b6f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:14.273638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2844034-1e32-4a10-8a5e-6bfd27c1b6f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:14.275286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2844034-1e32-4a10-8a5e-6bfd27c1b6f9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:14.293791 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 208/413] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:14.301143 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2844034-1e32-4a10-8a5e-6bfd27c1b6f9 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:14.322313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f75647-c579-4581-9a74-65b950c7b232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:14.322936 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f75647-c579-4581-9a74-65b950c7b232 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:14.323279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f75647-c579-4581-9a74-65b950c7b232 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:14.323554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f75647-c579-4581-9a74-65b950c7b232 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:14.324255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f75647-c579-4581-9a74-65b950c7b232 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:14.326255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f75647-c579-4581-9a74-65b950c7b232 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:14.327721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f75647-c579-4581-9a74-65b950c7b232 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:14.334354 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:14.350965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 206/414] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:14.356655 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f75647-c579-4581-9a74-65b950c7b232 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:14.486820 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:14.490170 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f75647-c579-4581-9a74-65b950c7b232 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:14.524039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2844034-1e32-4a10-8a5e-6bfd27c1b6f9 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:14.568718 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6850a55-4943-442e-b202-7e10e02cd953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:14.569671 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6850a55-4943-442e-b202-7e10e02cd953 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:14.570094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6850a55-4943-442e-b202-7e10e02cd953 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:14.570510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6850a55-4943-442e-b202-7e10e02cd953 None admin] PATH_INFO: `/v3/projects/f28d337589024cc88184355bd0ee8572/users/6ff451e43cea4bb0a5fd73491a05bf82/roles/8adf2afafb0446a6992e9aeb380f645d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:14.576038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05cba67-e8f6-415e-925e-818a8f116fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:14.576775 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05cba67-e8f6-415e-925e-818a8f116fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:14.577152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05cba67-e8f6-415e-925e-818a8f116fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:14.577614 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05cba67-e8f6-415e-925e-818a8f116fc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:14.590257 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05cba67-e8f6-415e-925e-818a8f116fc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:14.590257 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05cba67-e8f6-415e-925e-818a8f116fc1 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:14.603894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6850a55-4943-442e-b202-7e10e02cd953 None admin] RBAC: Authorizing `identity:create_grant(project_id=f28d337589024cc88184355bd0ee8572, user_id=6ff451e43cea4bb0a5fd73491a05bf82, role_id=8adf2afafb0446a6992e9aeb380f645d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:14.605446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6850a55-4943-442e-b202-7e10e02cd953 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-c05cba67-e8f6-415e-925e-818a8f116fc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:14.609695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:14.611156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:14.612224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:14.612224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:14.612224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:14.612224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:14.612224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:14.612224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:14.612224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:14.612224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:14.614481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 209/415] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:14.621660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05cba67-e8f6-415e-925e-818a8f116fc1 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:14.631346 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 207/416] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:12:14 2022] PUT /identity/v3/projects/f28d337589024cc88184355bd0ee8572/users/6ff451e43cea4bb0a5fd73491a05bf82/roles/8adf2afafb0446a6992e9aeb380f645d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:12:14.714412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:14.749663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05cba67-e8f6-415e-925e-818a8f116fc1 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:14.791601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd643395-344f-4943-9900-1ee81c0b3b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:14.792411 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd643395-344f-4943-9900-1ee81c0b3b0b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:14.792797 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd643395-344f-4943-9900-1ee81c0b3b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:14.793173 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd643395-344f-4943-9900-1ee81c0b3b0b None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:14.794002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd643395-344f-4943-9900-1ee81c0b3b0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:14.794416 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd643395-344f-4943-9900-1ee81c0b3b0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:14.796164 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd643395-344f-4943-9900-1ee81c0b3b0b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:14.810881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 210/417] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:14.817119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6850a55-4943-442e-b202-7e10e02cd953 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:14.928400 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6850a55-4943-442e-b202-7e10e02cd953 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:14.964237 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb28001-6484-48cc-a339-7ebb8e588c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:14.964923 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28001-6484-48cc-a339-7ebb8e588c56 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:14.965205 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28001-6484-48cc-a339-7ebb8e588c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:14.965477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28001-6484-48cc-a339-7ebb8e588c56 None admin] PATH_INFO: `/v3/projects/f28d337589024cc88184355bd0ee8572/users/eb7a43e284e1401688b2e202c67a5eb5/roles/8adf2afafb0446a6992e9aeb380f645d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:14.980983 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb28001-6484-48cc-a339-7ebb8e588c56 None admin] RBAC: Authorizing `identity:create_grant(project_id=f28d337589024cc88184355bd0ee8572, user_id=eb7a43e284e1401688b2e202c67a5eb5, role_id=8adf2afafb0446a6992e9aeb380f645d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:14.982461 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb28001-6484-48cc-a339-7ebb8e588c56 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:14.992689 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:14.997408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 208/418] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:12:14 2022] PUT /identity/v3/projects/f28d337589024cc88184355bd0ee8572/users/eb7a43e284e1401688b2e202c67a5eb5/roles/8adf2afafb0446a6992e9aeb380f645d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:12:15.737374 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d915b8d-0b0d-438f-b12a-521567fff72d None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:15.737834 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d915b8d-0b0d-438f-b12a-521567fff72d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:15.737925 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d915b8d-0b0d-438f-b12a-521567fff72d None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:15.739535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 211/419] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:15.746099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56221763-1e53-4c23-9c4d-89795e8186df None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:15.746384 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56221763-1e53-4c23-9c4d-89795e8186df None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:15.746660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56221763-1e53-4c23-9c4d-89795e8186df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:15.765227 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-56221763-1e53-4c23-9c4d-89795e8186df None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:15.766484 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-56221763-1e53-4c23-9c4d-89795e8186df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:15.842779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 209/420] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:16.054757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1054016-115f-499c-a5b6-20252f30f081 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:16.055158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1054016-115f-499c-a5b6-20252f30f081 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:16.055357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1054016-115f-499c-a5b6-20252f30f081 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:16.056948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 212/421] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:16.068380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d1e7b-2f8d-428d-8a1d-aef469b036d8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:16.068671 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d1e7b-2f8d-428d-8a1d-aef469b036d8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:16.068955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d1e7b-2f8d-428d-8a1d-aef469b036d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:16.096243 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-bc3d1e7b-2f8d-428d-8a1d-aef469b036d8 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:16.097475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bc3d1e7b-2f8d-428d-8a1d-aef469b036d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:16.189007 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 210/422] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:16.222549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1054016-115f-499c-a5b6-20252f30f081 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:16.351249 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1054016-115f-499c-a5b6-20252f30f081 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:16.396787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bef38-f1cf-4dde-869f-e5855baffb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:16.397782 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bef38-f1cf-4dde-869f-e5855baffb8c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:16.398258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bef38-f1cf-4dde-869f-e5855baffb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:16.398747 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bef38-f1cf-4dde-869f-e5855baffb8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:16.404405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:16.419958 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bef38-f1cf-4dde-869f-e5855baffb8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-810bef38-f1cf-4dde-869f-e5855baffb8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:16.425226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:16.426154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:16.426912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:16.426912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:16.426912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:16.426912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:16.426912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:16.426912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:16.426912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:16.426912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:16.426912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:16.426912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:16.426912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:16.426912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:16.430198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 213/423] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:12:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:16.436326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d1e7b-2f8d-428d-8a1d-aef469b036d8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:16.499675 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bef38-f1cf-4dde-869f-e5855baffb8c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:16.551451 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:16.631514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d1e7b-2f8d-428d-8a1d-aef469b036d8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:16.674282 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bef38-f1cf-4dde-869f-e5855baffb8c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:16.713545 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996a279-fba4-4b20-865b-32d7bb88ea2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:16.714521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996a279-fba4-4b20-865b-32d7bb88ea2d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:16.715000 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996a279-fba4-4b20-865b-32d7bb88ea2d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:16.715557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996a279-fba4-4b20-865b-32d7bb88ea2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:16.715987 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97a510-169f-4ac6-b277-1894ded9b8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:16.716629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97a510-169f-4ac6-b277-1894ded9b8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:16.716926 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97a510-169f-4ac6-b277-1894ded9b8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:16.717197 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97a510-169f-4ac6-b277-1894ded9b8b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:16.717911 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97a510-169f-4ac6-b277-1894ded9b8b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:16.718199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97a510-169f-4ac6-b277-1894ded9b8b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:16.720682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97a510-169f-4ac6-b277-1894ded9b8b1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:16.733350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996a279-fba4-4b20-865b-32d7bb88ea2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:16.734669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 211/424] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Oct 14 02:12:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a996a279-fba4-4b20-865b-32d7bb88ea2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:16.740201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:16.741442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:16.742230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:16.742230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:16.742230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:16.742230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:16.742230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:16.742230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:16.742230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:16.742230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:16.742230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:16.742230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:16.742230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:16.742230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:16.745493 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 214/425] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:12:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:16.750322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97a510-169f-4ac6-b277-1894ded9b8b1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:16.752556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996a279-fba4-4b20-865b-32d7bb88ea2d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:16.827474 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:16.876778 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996a279-fba4-4b20-865b-32d7bb88ea2d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:16.917430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97a510-169f-4ac6-b277-1894ded9b8b1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:16.918406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187151b5-bc8e-4b58-a357-12224a3f59f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:16.920407 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187151b5-bc8e-4b58-a357-12224a3f59f8 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:16.920903 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187151b5-bc8e-4b58-a357-12224a3f59f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:16.921435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187151b5-bc8e-4b58-a357-12224a3f59f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:16.922967 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187151b5-bc8e-4b58-a357-12224a3f59f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:16.923496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187151b5-bc8e-4b58-a357-12224a3f59f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:16.925591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187151b5-bc8e-4b58-a357-12224a3f59f8 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:16.937394 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 215/426] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Oct 14 02:12:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:16.942464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94df561-433b-4f33-a1f4-d23d78534545 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:16.942464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94df561-433b-4f33-a1f4-d23d78534545 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:16.942464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94df561-433b-4f33-a1f4-d23d78534545 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:16.947145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 216/427] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:16.948463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94df561-433b-4f33-a1f4-d23d78534545 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:16.949342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:16.968191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4d843-6c81-4e0d-a190-328c85185bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:16.969386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d843-6c81-4e0d-a190-328c85185bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:16.969949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d843-6c81-4e0d-a190-328c85185bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:16.970423 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d843-6c81-4e0d-a190-328c85185bd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:16.985500 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4d843-6c81-4e0d-a190-328c85185bd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:16.987539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4d843-6c81-4e0d-a190-328c85185bd7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d8e4d843-6c81-4e0d-a190-328c85185bd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:17.007271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:17.008209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:17.009032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:17.009032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:17.009032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:17.012814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 212/428] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:17.014560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466fc6e-d3b1-4ab3-9be0-9958fff39645 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:17.015134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466fc6e-d3b1-4ab3-9be0-9958fff39645 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:17.015611 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466fc6e-d3b1-4ab3-9be0-9958fff39645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:17.038531 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1466fc6e-d3b1-4ab3-9be0-9958fff39645 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:17.040298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1466fc6e-d3b1-4ab3-9be0-9958fff39645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:17.069990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94df561-433b-4f33-a1f4-d23d78534545 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:17.095017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:17.119136 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 213/429] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:17.121047 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466fc6e-d3b1-4ab3-9be0-9958fff39645 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:17.126140 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb108ed6-acfa-4bdc-a996-b3ee7a066a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:17.128042 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb108ed6-acfa-4bdc-a996-b3ee7a066a2b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:17.128589 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb108ed6-acfa-4bdc-a996-b3ee7a066a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:17.129212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb108ed6-acfa-4bdc-a996-b3ee7a066a2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:17.152116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb108ed6-acfa-4bdc-a996-b3ee7a066a2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:17.156118 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb108ed6-acfa-4bdc-a996-b3ee7a066a2b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-fb108ed6-acfa-4bdc-a996-b3ee7a066a2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:17.188717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:17.189729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:17.190554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:17.190554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:17.190554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:17.190554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 217/430] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:17.207271 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb108ed6-acfa-4bdc-a996-b3ee7a066a2b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:17.278144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466fc6e-d3b1-4ab3-9be0-9958fff39645 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:17.282377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:17.325781 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63776144-5754-49f3-bd8e-4cb736134cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:17.326905 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63776144-5754-49f3-bd8e-4cb736134cef None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:17.328206 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63776144-5754-49f3-bd8e-4cb736134cef None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:17.329010 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63776144-5754-49f3-bd8e-4cb736134cef None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:17.331186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63776144-5754-49f3-bd8e-4cb736134cef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:17.332030 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63776144-5754-49f3-bd8e-4cb736134cef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:17.334332 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63776144-5754-49f3-bd8e-4cb736134cef None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:17.364456 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 214/431] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:17.374106 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63776144-5754-49f3-bd8e-4cb736134cef None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:17.418203 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb108ed6-acfa-4bdc-a996-b3ee7a066a2b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:17.460443 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde805e0-99d9-4a24-a937-b6cb46108e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:17.460999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde805e0-99d9-4a24-a937-b6cb46108e59 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:17.461240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde805e0-99d9-4a24-a937-b6cb46108e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:17.461865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde805e0-99d9-4a24-a937-b6cb46108e59 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:17.462504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde805e0-99d9-4a24-a937-b6cb46108e59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:17.463137 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde805e0-99d9-4a24-a937-b6cb46108e59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:17.464512 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde805e0-99d9-4a24-a937-b6cb46108e59 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:17.483465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 218/432] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:17.483465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba23899-678c-4b94-8723-4da84ebd122c None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:17.483465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba23899-678c-4b94-8723-4da84ebd122c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:17.483465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba23899-678c-4b94-8723-4da84ebd122c None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:17.483465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 219/433] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:17.483465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba23899-678c-4b94-8723-4da84ebd122c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:17.494869 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63776144-5754-49f3-bd8e-4cb736134cef None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:17.513641 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:17.535667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09b54b-c797-4b0a-944c-80b9db09ed60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:17.536328 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09b54b-c797-4b0a-944c-80b9db09ed60 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:17.536602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09b54b-c797-4b0a-944c-80b9db09ed60 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:17.536871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09b54b-c797-4b0a-944c-80b9db09ed60 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:17.546208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09b54b-c797-4b0a-944c-80b9db09ed60 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:17.548623 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09b54b-c797-4b0a-944c-80b9db09ed60 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-cb09b54b-c797-4b0a-944c-80b9db09ed60 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:17.561186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:17.562099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:17.568200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:17.568200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:17.568200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:17.568200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:17.568200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:17.568200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:17.568200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:17.568200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:17.568200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:12:17.568200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:17.568200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 215/434] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Oct 14 02:12:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:17.568200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09b54b-c797-4b0a-944c-80b9db09ed60 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:17.591651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:17.602512 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba23899-678c-4b94-8723-4da84ebd122c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:17.638779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc33b36-c3cd-49d2-96c3-e563a98ce334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:17.639909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc33b36-c3cd-49d2-96c3-e563a98ce334 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:17.640182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc33b36-c3cd-49d2-96c3-e563a98ce334 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:17.640452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc33b36-c3cd-49d2-96c3-e563a98ce334 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:17.651694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc33b36-c3cd-49d2-96c3-e563a98ce334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-4cc33b36-c3cd-49d2-96c3-e563a98ce334 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:17.656421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:17.657366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:17.658123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:17.658123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:17.658123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:17.658123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:17.658123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:17.658123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:17.658123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:17.658123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:17.658123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:17.658123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:17.658123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:17.658123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:17.658123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 220/435] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:12:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:17.658864 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efecec-3822-4ca3-a567-b8b6fd32b3a8 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:17.659147 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efecec-3822-4ca3-a567-b8b6fd32b3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:17.659409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efecec-3822-4ca3-a567-b8b6fd32b3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:17.675034 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09b54b-c797-4b0a-944c-80b9db09ed60 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:17.676927 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-55efecec-3822-4ca3-a567-b8b6fd32b3a8 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:17.678098 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-55efecec-3822-4ca3-a567-b8b6fd32b3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:17.709410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f375dc-b49e-4a58-9fcd-8ad6571eecbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:17.710054 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f375dc-b49e-4a58-9fcd-8ad6571eecbe None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:17.710331 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f375dc-b49e-4a58-9fcd-8ad6571eecbe None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:17.710659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f375dc-b49e-4a58-9fcd-8ad6571eecbe None admin] PATH_INFO: `/v3/users/swift` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:17.717839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f375dc-b49e-4a58-9fcd-8ad6571eecbe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:17.719322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f375dc-b49e-4a58-9fcd-8ad6571eecbe None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-44f375dc-b49e-4a58-9fcd-8ad6571eecbe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:17.730240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:17.731240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:17.732102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:17.732102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:17.732102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:17.732102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:17.732102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:17.732102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:17.732102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:17.732102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:17.732102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:12:17.732102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:17.735782 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 216/436] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:12:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:17.736984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f375dc-b49e-4a58-9fcd-8ad6571eecbe None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:17.746693 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 221/437] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:17.748766 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334c175-caa7-4016-b289-0a0595d3a9c5 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:17.749051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334c175-caa7-4016-b289-0a0595d3a9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:17.749328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334c175-caa7-4016-b289-0a0595d3a9c5 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:17.754841 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 222/438] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:17.754841 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334c175-caa7-4016-b289-0a0595d3a9c5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:17.784063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:17.837068 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:17.855918 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f375dc-b49e-4a58-9fcd-8ad6571eecbe None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:17.866485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334c175-caa7-4016-b289-0a0595d3a9c5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:17.894251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01379311-9778-49da-be99-71678ee62c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:17.894899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01379311-9778-49da-be99-71678ee62c6b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:17.895209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01379311-9778-49da-be99-71678ee62c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:17.895483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01379311-9778-49da-be99-71678ee62c6b None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:17.896386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01379311-9778-49da-be99-71678ee62c6b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:17.897062 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01379311-9778-49da-be99-71678ee62c6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:17.899000 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01379311-9778-49da-be99-71678ee62c6b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:17.901381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec5bca-3d51-4595-8142-5adf3c44326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:17.901958 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec5bca-3d51-4595-8142-5adf3c44326f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:17.902233 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec5bca-3d51-4595-8142-5adf3c44326f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:17.902503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec5bca-3d51-4595-8142-5adf3c44326f None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:17.903197 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec5bca-3d51-4595-8142-5adf3c44326f None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:17.903500 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec5bca-3d51-4595-8142-5adf3c44326f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:17.904860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec5bca-3d51-4595-8142-5adf3c44326f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:17.918077 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 223/439] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Oct 14 02:12:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:17.918077 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec5bca-3d51-4595-8142-5adf3c44326f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:17.937832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 217/440] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Oct 14 02:12:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:17.940313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a45ab8-61f6-4981-b6e5-51e41b622569 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:17.940601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a45ab8-61f6-4981-b6e5-51e41b622569 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:17.940870 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a45ab8-61f6-4981-b6e5-51e41b622569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:17.958632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c9a45ab8-61f6-4981-b6e5-51e41b622569 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:17.959848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c9a45ab8-61f6-4981-b6e5-51e41b622569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:18.011722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:18.030007 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 218/441] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:18.030771 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a45ab8-61f6-4981-b6e5-51e41b622569 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:18.033459 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec5bca-3d51-4595-8142-5adf3c44326f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:18.068913 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304437fe-eac7-43fe-acff-74bd234b331c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:18.069518 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304437fe-eac7-43fe-acff-74bd234b331c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:18.069794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304437fe-eac7-43fe-acff-74bd234b331c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:18.070062 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304437fe-eac7-43fe-acff-74bd234b331c None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:18.070784 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304437fe-eac7-43fe-acff-74bd234b331c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:18.071114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304437fe-eac7-43fe-acff-74bd234b331c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:18.072533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304437fe-eac7-43fe-acff-74bd234b331c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:18.091109 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 224/442] 10.222.0.18 () {56 vars in 1135 bytes} [Fri Oct 14 02:12:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:18.096788 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304437fe-eac7-43fe-acff-74bd234b331c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:18.104092 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:18.141707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a45ab8-61f6-4981-b6e5-51e41b622569 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:18.182557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795850ad-348b-46a7-b908-87d31ab4b77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:18.183251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795850ad-348b-46a7-b908-87d31ab4b77c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:18.183534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795850ad-348b-46a7-b908-87d31ab4b77c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:18.183807 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795850ad-348b-46a7-b908-87d31ab4b77c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:18.194996 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795850ad-348b-46a7-b908-87d31ab4b77c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:18.196490 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795850ad-348b-46a7-b908-87d31ab4b77c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:18.196589 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304437fe-eac7-43fe-acff-74bd234b331c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-795850ad-348b-46a7-b908-87d31ab4b77c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:18.212323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:18.213273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:18.214095 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:18.214095 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:18.214095 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:18.214095 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 219/443] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:18.214298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795850ad-348b-46a7-b908-87d31ab4b77c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:18.239011 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cdbe0-157f-461b-9033-900bee950213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:18.239650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cdbe0-157f-461b-9033-900bee950213 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:18.239934 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cdbe0-157f-461b-9033-900bee950213 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:18.240207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cdbe0-157f-461b-9033-900bee950213 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:18.251842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:18.313647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cdbe0-157f-461b-9033-900bee950213 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:18.315004 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cdbe0-157f-461b-9033-900bee950213 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:18.317785 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795850ad-348b-46a7-b908-87d31ab4b77c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:18.340464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:18.353074 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428a8a9-1dc4-42b9-9fbc-f8ad9336a709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:18.353706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428a8a9-1dc4-42b9-9fbc-f8ad9336a709 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:18.353985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428a8a9-1dc4-42b9-9fbc-f8ad9336a709 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:18.354260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428a8a9-1dc4-42b9-9fbc-f8ad9336a709 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:18.364908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428a8a9-1dc4-42b9-9fbc-f8ad9336a709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-8428a8a9-1dc4-42b9-9fbc-f8ad9336a709 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:18.370188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:18.371122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:18.371897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:18.371897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:18.371897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:18.371897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:18.371897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:18.371897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:18.371897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:18.371897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:18.371897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:18.371897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:18.371897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:18.371897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:18.371897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 220/444] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:12:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:18.372793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428a8a9-1dc4-42b9-9fbc-f8ad9336a709 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:18.418704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 225/445] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:12:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:12:18.419583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cdbe0-157f-461b-9033-900bee950213 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:18.465082 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:18.473211 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428a8a9-1dc4-42b9-9fbc-f8ad9336a709 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:18.505422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e361abf-0bbe-4007-951a-ad9662e564f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:18.506050 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e361abf-0bbe-4007-951a-ad9662e564f2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:18.506683 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e361abf-0bbe-4007-951a-ad9662e564f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:18.506953 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e361abf-0bbe-4007-951a-ad9662e564f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:18.519144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e361abf-0bbe-4007-951a-ad9662e564f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:18.520602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e361abf-0bbe-4007-951a-ad9662e564f2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:18.524549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cdbe0-157f-461b-9033-900bee950213 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-1e361abf-0bbe-4007-951a-ad9662e564f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:18.535739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:18.536596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:18.537391 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:18.537391 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:18.537391 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:18.537391 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 221/446] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:18.537639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e361abf-0bbe-4007-951a-ad9662e564f2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:18.560424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875cfe7-8982-49ed-b0e6-74cd4f3632aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:18.561032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875cfe7-8982-49ed-b0e6-74cd4f3632aa None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:18.561311 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875cfe7-8982-49ed-b0e6-74cd4f3632aa None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:18.561582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875cfe7-8982-49ed-b0e6-74cd4f3632aa None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:18.562216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875cfe7-8982-49ed-b0e6-74cd4f3632aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:18.562518 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875cfe7-8982-49ed-b0e6-74cd4f3632aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:18.564079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875cfe7-8982-49ed-b0e6-74cd4f3632aa None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:18.571686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:18.575584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 226/447] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:18.576835 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875cfe7-8982-49ed-b0e6-74cd4f3632aa None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:18.637406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e361abf-0bbe-4007-951a-ad9662e564f2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:18.670304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707b854-9522-4fca-af90-7e56bb56ea27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:18.670938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707b854-9522-4fca-af90-7e56bb56ea27 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:18.671251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707b854-9522-4fca-af90-7e56bb56ea27 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:18.671526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707b854-9522-4fca-af90-7e56bb56ea27 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:18.675455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:18.678453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875cfe7-8982-49ed-b0e6-74cd4f3632aa None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:18.682235 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707b854-9522-4fca-af90-7e56bb56ea27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d707b854-9522-4fca-af90-7e56bb56ea27 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:18.687491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:18.688553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:18.689411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:18.689411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:18.689411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:18.689411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:18.689411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:18.689411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:18.689411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:18.689411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:18.689411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:18.689411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:18.689411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:18.689411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:18.689411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 222/448] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:12:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:18.690041 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707b854-9522-4fca-af90-7e56bb56ea27 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:18.714027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963a3e9-38ac-4950-a1cd-75e3064a25cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:18.714627 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963a3e9-38ac-4950-a1cd-75e3064a25cc None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:18.714911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963a3e9-38ac-4950-a1cd-75e3064a25cc None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:18.715209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963a3e9-38ac-4950-a1cd-75e3064a25cc None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:18.716097 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963a3e9-38ac-4950-a1cd-75e3064a25cc None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:18.716207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963a3e9-38ac-4950-a1cd-75e3064a25cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:18.717496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963a3e9-38ac-4950-a1cd-75e3064a25cc None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:18.728914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 227/449] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Oct 14 02:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:18.734251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963a3e9-38ac-4950-a1cd-75e3064a25cc None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:18.786726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:18.792843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707b854-9522-4fca-af90-7e56bb56ea27 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:18.826452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab973bc-e7a1-45d8-938a-0ae086699a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:18.827017 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab973bc-e7a1-45d8-938a-0ae086699a9b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:18.827330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab973bc-e7a1-45d8-938a-0ae086699a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:18.827606 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab973bc-e7a1-45d8-938a-0ae086699a9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:18.833166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963a3e9-38ac-4950-a1cd-75e3064a25cc None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:18.839295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab973bc-e7a1-45d8-938a-0ae086699a9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:18.840752 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab973bc-e7a1-45d8-938a-0ae086699a9b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-cab973bc-e7a1-45d8-938a-0ae086699a9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:18.854632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:18.855772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:18.856682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:18.856682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:18.856682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:18.856682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 223/450] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:18.857057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab973bc-e7a1-45d8-938a-0ae086699a9b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:18.869053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33338f39-51d2-4be5-8506-0a639eb543ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:18.869658 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33338f39-51d2-4be5-8506-0a639eb543ac None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:18.869932 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33338f39-51d2-4be5-8506-0a639eb543ac None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:18.870200 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33338f39-51d2-4be5-8506-0a639eb543ac None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:18.870831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33338f39-51d2-4be5-8506-0a639eb543ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:18.871156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33338f39-51d2-4be5-8506-0a639eb543ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:18.872753 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33338f39-51d2-4be5-8506-0a639eb543ac None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:18.884926 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 228/451] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:18.885722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33338f39-51d2-4be5-8506-0a639eb543ac None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:18.909416 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:18.956823 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab973bc-e7a1-45d8-938a-0ae086699a9b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:18.983499 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33338f39-51d2-4be5-8506-0a639eb543ac None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:18.990902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d4875-d6ba-4765-b80b-0cdff1b13215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:18.991553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d4875-d6ba-4765-b80b-0cdff1b13215 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:18.991831 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d4875-d6ba-4765-b80b-0cdff1b13215 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:18.992105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d4875-d6ba-4765-b80b-0cdff1b13215 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:18.999580 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d4875-d6ba-4765-b80b-0cdff1b13215 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:19.001054 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d4875-d6ba-4765-b80b-0cdff1b13215 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-e41d4875-d6ba-4765-b80b-0cdff1b13215 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:19.011850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:19.012706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:19.019322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:19.019322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:19.019322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:19.019322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:19.019322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:19.019322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:19.019322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:19.019322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:19.019322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 14 02:12:19.019322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:19.019322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 224/452] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:12:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:19.019322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d4875-d6ba-4765-b80b-0cdff1b13215 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:19.020069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805bbe6-f147-41bd-98cb-e9d53a458659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:19.020069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805bbe6-f147-41bd-98cb-e9d53a458659 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:19.020069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805bbe6-f147-41bd-98cb-e9d53a458659 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:19.020069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805bbe6-f147-41bd-98cb-e9d53a458659 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:19.020358 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805bbe6-f147-41bd-98cb-e9d53a458659 None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:19.020358 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805bbe6-f147-41bd-98cb-e9d53a458659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:19.021234 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805bbe6-f147-41bd-98cb-e9d53a458659 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:19.024857 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:19.029182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 229/453] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Oct 14 02:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:19.029979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805bbe6-f147-41bd-98cb-e9d53a458659 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:19.112537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:19.113307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d4875-d6ba-4765-b80b-0cdff1b13215 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:19.129038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805bbe6-f147-41bd-98cb-e9d53a458659 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:19.146125 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3569f-94da-44ac-a56d-2d3896a1e347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:19.146757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3569f-94da-44ac-a56d-2d3896a1e347 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:19.147038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3569f-94da-44ac-a56d-2d3896a1e347 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:19.147339 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3569f-94da-44ac-a56d-2d3896a1e347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:19.158622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3569f-94da-44ac-a56d-2d3896a1e347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:19.160109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3569f-94da-44ac-a56d-2d3896a1e347 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:19.161717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e3815-1f06-49aa-9334-cfe3c7dbd540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:19.162317 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e3815-1f06-49aa-9334-cfe3c7dbd540 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:19.162587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e3815-1f06-49aa-9334-cfe3c7dbd540 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:19.162851 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e3815-1f06-49aa-9334-cfe3c7dbd540 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:19.163509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e3815-1f06-49aa-9334-cfe3c7dbd540 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:19.163806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e3815-1f06-49aa-9334-cfe3c7dbd540 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:19.165141 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e3815-1f06-49aa-9334-cfe3c7dbd540 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-ddf3569f-94da-44ac-a56d-2d3896a1e347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:19.174510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:19.175551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:19.176365 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:19.176365 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:19.176365 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:19.176365 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 225/454] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:19.176365 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3569f-94da-44ac-a56d-2d3896a1e347 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:19.181584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 230/455] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:19.181584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e3815-1f06-49aa-9334-cfe3c7dbd540 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:19.237465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:19.275078 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e3815-1f06-49aa-9334-cfe3c7dbd540 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:19.278055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3569f-94da-44ac-a56d-2d3896a1e347 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:19.308062 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8edb28-f62c-44c1-9b5f-60963d8f2abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:19.308666 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8edb28-f62c-44c1-9b5f-60963d8f2abe None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:19.308946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8edb28-f62c-44c1-9b5f-60963d8f2abe None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:19.309217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8edb28-f62c-44c1-9b5f-60963d8f2abe None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:19.309938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8edb28-f62c-44c1-9b5f-60963d8f2abe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:19.310243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8edb28-f62c-44c1-9b5f-60963d8f2abe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:19.310668 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabd424-0bd6-40ac-8ff0-5f8059bf6995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:19.311324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd424-0bd6-40ac-8ff0-5f8059bf6995 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:19.311599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd424-0bd6-40ac-8ff0-5f8059bf6995 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:19.311719 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8edb28-f62c-44c1-9b5f-60963d8f2abe None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:19.311865 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd424-0bd6-40ac-8ff0-5f8059bf6995 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:19.319360 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabd424-0bd6-40ac-8ff0-5f8059bf6995 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:19.320777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabd424-0bd6-40ac-8ff0-5f8059bf6995 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:19.328971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 231/456] 10.222.0.18 () {56 vars in 1137 bytes} [Fri Oct 14 02:12:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:19.329775 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8edb28-f62c-44c1-9b5f-60963d8f2abe None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-3fabd424-0bd6-40ac-8ff0-5f8059bf6995 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:19.331904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:19.332679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:19.333477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:19.333477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:19.333477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:19.333477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:19.333477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:19.333477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:19.333477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:19.333477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:19.333477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 226/457] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:19.334323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabd424-0bd6-40ac-8ff0-5f8059bf6995 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:19.334427 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:19.429872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabd424-0bd6-40ac-8ff0-5f8059bf6995 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:19.431092 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8edb28-f62c-44c1-9b5f-60963d8f2abe None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:19.449177 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:19.464413 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae2248-27f2-4f47-b099-5d291d29c808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:19.465019 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae2248-27f2-4f47-b099-5d291d29c808 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:19.465116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca61ba-b007-4c34-a472-a7fd1e133071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:19.465296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae2248-27f2-4f47-b099-5d291d29c808 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:19.465567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae2248-27f2-4f47-b099-5d291d29c808 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:19.465650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca61ba-b007-4c34-a472-a7fd1e133071 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:19.465911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca61ba-b007-4c34-a472-a7fd1e133071 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:19.466179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca61ba-b007-4c34-a472-a7fd1e133071 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:19.466262 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae2248-27f2-4f47-b099-5d291d29c808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:19.466485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae2248-27f2-4f47-b099-5d291d29c808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:19.467877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae2248-27f2-4f47-b099-5d291d29c808 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:19.476411 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca61ba-b007-4c34-a472-a7fd1e133071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:19.477836 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca61ba-b007-4c34-a472-a7fd1e133071 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:19.479097 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 227/458] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:19.479883 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae2248-27f2-4f47-b099-5d291d29c808 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-1aca61ba-b007-4c34-a472-a7fd1e133071 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:19.492219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:19.493152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:19.494490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:19.494490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:19.494490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:19.494490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 232/459] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:19.494490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca61ba-b007-4c34-a472-a7fd1e133071 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:19.552156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:19.583011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae2248-27f2-4f47-b099-5d291d29c808 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:19.597001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca61ba-b007-4c34-a472-a7fd1e133071 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:19.616305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a974c-4bc5-40a4-a618-037524a7bc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:19.616960 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a974c-4bc5-40a4-a618-037524a7bc8e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:19.617237 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a974c-4bc5-40a4-a618-037524a7bc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:19.617505 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a974c-4bc5-40a4-a618-037524a7bc8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:19.624206 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a974c-4bc5-40a4-a618-037524a7bc8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:19.626168 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a974c-4bc5-40a4-a618-037524a7bc8e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:19.630895 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f36354-4a32-4a82-bc3f-31e4e5848127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:19.631547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f36354-4a32-4a82-bc3f-31e4e5848127 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:19.631822 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f36354-4a32-4a82-bc3f-31e4e5848127 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:19.632086 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f36354-4a32-4a82-bc3f-31e4e5848127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-be1a974c-4bc5-40a4-a618-037524a7bc8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:19.636981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:19.637784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:19.638588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:19.638588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:19.638588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:19.638588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:19.638588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:19.638588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:19.638588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:19.638588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:19.638588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 228/460] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:19.639053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a974c-4bc5-40a4-a618-037524a7bc8e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:19.683616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:19.702219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f36354-4a32-4a82-bc3f-31e4e5848127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:19.703789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f36354-4a32-4a82-bc3f-31e4e5848127 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:19.742282 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a974c-4bc5-40a4-a618-037524a7bc8e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:19.757937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:19.776528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97258b-f00e-43b7-9869-30fdf4b26064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:19.777147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97258b-f00e-43b7-9869-30fdf4b26064 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:19.777428 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97258b-f00e-43b7-9869-30fdf4b26064 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:19.777703 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97258b-f00e-43b7-9869-30fdf4b26064 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:19.778392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97258b-f00e-43b7-9869-30fdf4b26064 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:19.778698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97258b-f00e-43b7-9869-30fdf4b26064 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:19.780187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97258b-f00e-43b7-9869-30fdf4b26064 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:19.792797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 229/461] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:19.793596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97258b-f00e-43b7-9869-30fdf4b26064 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:19.805166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 233/462] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:12:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:12:19.806007 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f36354-4a32-4a82-bc3f-31e4e5848127 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:19.889554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:19.892880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97258b-f00e-43b7-9869-30fdf4b26064 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:19.903306 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f36354-4a32-4a82-bc3f-31e4e5848127 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:19.926929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136378c5-65b9-4821-ad11-20e0519a1413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:19.927894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136378c5-65b9-4821-ad11-20e0519a1413 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:19.928171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136378c5-65b9-4821-ad11-20e0519a1413 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:19.928439 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136378c5-65b9-4821-ad11-20e0519a1413 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:19.935277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136378c5-65b9-4821-ad11-20e0519a1413 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:19.936646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136378c5-65b9-4821-ad11-20e0519a1413 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:19.937007 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6d52e-3f78-470f-b49f-07bc854c508b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:19.937606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6d52e-3f78-470f-b49f-07bc854c508b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:19.938044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6d52e-3f78-470f-b49f-07bc854c508b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:19.938329 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6d52e-3f78-470f-b49f-07bc854c508b None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:19.938969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6d52e-3f78-470f-b49f-07bc854c508b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:19.939300 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6d52e-3f78-470f-b49f-07bc854c508b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:19.940655 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6d52e-3f78-470f-b49f-07bc854c508b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-136378c5-65b9-4821-ad11-20e0519a1413 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:19.948238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:19.949082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:19.949929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:19.949929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:19.949929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:19.949929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:19.949929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:19.949929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:19.949929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:19.949929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:19.949929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 14 02:12:19.949929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:19.949929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 230/463] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:12:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:19.950469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136378c5-65b9-4821-ad11-20e0519a1413 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:19.955659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 234/464] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:19.956398 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6d52e-3f78-470f-b49f-07bc854c508b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:19.974606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:20.050117 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136378c5-65b9-4821-ad11-20e0519a1413 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:20.056845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6d52e-3f78-470f-b49f-07bc854c508b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:20.083924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae829d-7de5-4b2f-b9f4-d6760ad3e808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:20.084551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae829d-7de5-4b2f-b9f4-d6760ad3e808 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:20.084833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae829d-7de5-4b2f-b9f4-d6760ad3e808 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:20.085110 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae829d-7de5-4b2f-b9f4-d6760ad3e808 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:20.085801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae829d-7de5-4b2f-b9f4-d6760ad3e808 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:20.086107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae829d-7de5-4b2f-b9f4-d6760ad3e808 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:20.087588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae829d-7de5-4b2f-b9f4-d6760ad3e808 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:20.089403 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c71f8-8ab3-42a7-aecd-87453ce6ab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:20.090051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c71f8-8ab3-42a7-aecd-87453ce6ab47 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:20.090327 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c71f8-8ab3-42a7-aecd-87453ce6ab47 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:20.090598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c71f8-8ab3-42a7-aecd-87453ce6ab47 None admin] PATH_INFO: `/v3/projects/f28d337589024cc88184355bd0ee8572/users/cf0efd6d12ad458388e0c3574bbedc6a/roles/8adf2afafb0446a6992e9aeb380f645d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:20.099131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 231/465] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:20.099932 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae829d-7de5-4b2f-b9f4-d6760ad3e808 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:20.107635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c71f8-8ab3-42a7-aecd-87453ce6ab47 None admin] RBAC: Authorizing `identity:create_grant(project_id=f28d337589024cc88184355bd0ee8572, user_id=cf0efd6d12ad458388e0c3574bbedc6a, role_id=8adf2afafb0446a6992e9aeb380f645d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:20.109076 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c71f8-8ab3-42a7-aecd-87453ce6ab47 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:20.122949 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 235/466] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:12:19 2022] PUT /identity/v3/projects/f28d337589024cc88184355bd0ee8572/users/cf0efd6d12ad458388e0c3574bbedc6a/roles/8adf2afafb0446a6992e9aeb380f645d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:12:20.123780 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c71f8-8ab3-42a7-aecd-87453ce6ab47 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:20.144911 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:20.196949 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:20.214318 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae829d-7de5-4b2f-b9f4-d6760ad3e808 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:20.241203 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c71f8-8ab3-42a7-aecd-87453ce6ab47 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:20.253025 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59d7b5-4781-435f-bbf7-28201dc06e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:20.253685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59d7b5-4781-435f-bbf7-28201dc06e8b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:20.253963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59d7b5-4781-435f-bbf7-28201dc06e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:20.254246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59d7b5-4781-435f-bbf7-28201dc06e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:20.266541 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59d7b5-4781-435f-bbf7-28201dc06e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:20.268073 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59d7b5-4781-435f-bbf7-28201dc06e8b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:20.279970 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de3b70-a818-4c74-bc82-fbee3bbf45ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:20.280590 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de3b70-a818-4c74-bc82-fbee3bbf45ca None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:20.280867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de3b70-a818-4c74-bc82-fbee3bbf45ca None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:20.281139 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de3b70-a818-4c74-bc82-fbee3bbf45ca None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:20.282559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de3b70-a818-4c74-bc82-fbee3bbf45ca None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:20.282867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de3b70-a818-4c74-bc82-fbee3bbf45ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:20.284391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de3b70-a818-4c74-bc82-fbee3bbf45ca None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-1f59d7b5-4781-435f-bbf7-28201dc06e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:20.284861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:20.285712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:20.286532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:20.286532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:20.286532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:20.286532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 232/467] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:20.287458 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59d7b5-4781-435f-bbf7-28201dc06e8b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:20.305363 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 236/468] 10.222.0.18 () {56 vars in 1139 bytes} [Fri Oct 14 02:12:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:20.306370 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de3b70-a818-4c74-bc82-fbee3bbf45ca None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:20.369941 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:20.411094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59d7b5-4781-435f-bbf7-28201dc06e8b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:20.421021 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de3b70-a818-4c74-bc82-fbee3bbf45ca None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:20.447323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:20.449369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ee5a0-2012-4d5b-9d65-ff58f32fcc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:20.450009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ee5a0-2012-4d5b-9d65-ff58f32fcc38 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:20.450359 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ee5a0-2012-4d5b-9d65-ff58f32fcc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:20.450633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ee5a0-2012-4d5b-9d65-ff58f32fcc38 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:20.460950 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c965f7-0829-4b39-886c-117b8bc10216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:20.461570 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c965f7-0829-4b39-886c-117b8bc10216 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:20.461851 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c965f7-0829-4b39-886c-117b8bc10216 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:20.462126 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c965f7-0829-4b39-886c-117b8bc10216 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:20.462897 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c965f7-0829-4b39-886c-117b8bc10216 None admin] RBAC: Adding query filter params (role.id=8adf2afafb0446a6992e9aeb380f645d, scope.project.id=f28d337589024cc88184355bd0ee8572, user.id=de13be55b4034477baf4c029c1a79757) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:20.463159 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ee5a0-2012-4d5b-9d65-ff58f32fcc38 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:20.464954 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c965f7-0829-4b39-886c-117b8bc10216 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:20.465621 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c965f7-0829-4b39-886c-117b8bc10216 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:20.466035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ee5a0-2012-4d5b-9d65-ff58f32fcc38 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:20.472981 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 237/469] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:12:20 2022] GET /identity/v3/role_assignments?user.id=de13be55b4034477baf4c029c1a79757&scope.project.id=f28d337589024cc88184355bd0ee8572&role.id=8adf2afafb0446a6992e9aeb380f645d => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:20.473835 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c965f7-0829-4b39-886c-117b8bc10216 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-351ee5a0-2012-4d5b-9d65-ff58f32fcc38 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:20.490313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:20.491504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:20.492573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:20.492573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:20.492573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:20.492573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:20.492573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:20.492573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:20.492573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:20.492573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:20.492573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 02:12:20.492573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:20.494758 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 233/470] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:12:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:20.495888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ee5a0-2012-4d5b-9d65-ff58f32fcc38 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:20.616289 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ee5a0-2012-4d5b-9d65-ff58f32fcc38 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:20.638079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c965f7-0829-4b39-886c-117b8bc10216 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:20.651788 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:20.659370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239a4e3-eecd-43a6-abdd-ea807b6c17a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:20.660146 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239a4e3-eecd-43a6-abdd-ea807b6c17a9 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:20.660694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239a4e3-eecd-43a6-abdd-ea807b6c17a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:20.660857 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239a4e3-eecd-43a6-abdd-ea807b6c17a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:20.689609 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac5cac-97c3-4335-93ff-f12a05816906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:20.690228 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac5cac-97c3-4335-93ff-f12a05816906 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:20.690503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac5cac-97c3-4335-93ff-f12a05816906 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:20.690776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac5cac-97c3-4335-93ff-f12a05816906 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:20.691457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac5cac-97c3-4335-93ff-f12a05816906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:20.691809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac5cac-97c3-4335-93ff-f12a05816906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:20.693333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac5cac-97c3-4335-93ff-f12a05816906 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:20.708037 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 238/471] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:20.708869 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac5cac-97c3-4335-93ff-f12a05816906 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:20.756361 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239a4e3-eecd-43a6-abdd-ea807b6c17a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:20.758727 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239a4e3-eecd-43a6-abdd-ea807b6c17a9 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:20.761396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:20.823000 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac5cac-97c3-4335-93ff-f12a05816906 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:20.863331 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36e3cd-c036-4df5-bbdb-d5b65fd007c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:20.864181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36e3cd-c036-4df5-bbdb-d5b65fd007c1 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:20.864645 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36e3cd-c036-4df5-bbdb-d5b65fd007c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:20.865057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36e3cd-c036-4df5-bbdb-d5b65fd007c1 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:20.866107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36e3cd-c036-4df5-bbdb-d5b65fd007c1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:20.866556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36e3cd-c036-4df5-bbdb-d5b65fd007c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:20.868167 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36e3cd-c036-4df5-bbdb-d5b65fd007c1 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:20.889298 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 239/472] 10.222.0.18 () {56 vars in 1137 bytes} [Fri Oct 14 02:12:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:20.890426 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36e3cd-c036-4df5-bbdb-d5b65fd007c1 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:20.892109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 234/473] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:12:20.895295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239a4e3-eecd-43a6-abdd-ea807b6c17a9 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:20.927200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:21.001822 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36e3cd-c036-4df5-bbdb-d5b65fd007c1 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:21.004860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:21.005372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239a4e3-eecd-43a6-abdd-ea807b6c17a9 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:21.042729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d75f42-93ef-4518-84f4-ccb295b68f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:21.043494 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d75f42-93ef-4518-84f4-ccb295b68f0f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:21.043841 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d75f42-93ef-4518-84f4-ccb295b68f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:21.044183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d75f42-93ef-4518-84f4-ccb295b68f0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:21.047344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d2a5c-8003-4416-9604-d3f81fea1a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:21.048142 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d2a5c-8003-4416-9604-d3f81fea1a5b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:21.048578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d2a5c-8003-4416-9604-d3f81fea1a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:21.048996 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d2a5c-8003-4416-9604-d3f81fea1a5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:21.051575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d75f42-93ef-4518-84f4-ccb295b68f0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:21.053445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d75f42-93ef-4518-84f4-ccb295b68f0f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-d0d75f42-93ef-4518-84f4-ccb295b68f0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:21.067644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:21.068673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:21.069690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:21.069690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:21.069690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:21.069690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:21.069690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:21.069690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:21.069690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:21.069690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:21.075424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 240/474] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:21.076329 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d75f42-93ef-4518-84f4-ccb295b68f0f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:21.135088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d2a5c-8003-4416-9604-d3f81fea1a5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:21.136686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d2a5c-8003-4416-9604-d3f81fea1a5b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:21.179549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:21.196940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d75f42-93ef-4518-84f4-ccb295b68f0f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:21.240117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3df66-6e58-4c2e-af19-90afa4dc7441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:21.240759 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3df66-6e58-4c2e-af19-90afa4dc7441 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:21.241039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3df66-6e58-4c2e-af19-90afa4dc7441 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:21.241453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3df66-6e58-4c2e-af19-90afa4dc7441 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:21.254139 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3df66-6e58-4c2e-af19-90afa4dc7441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:21.255831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3df66-6e58-4c2e-af19-90afa4dc7441 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:21.270395 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 235/475] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-38a3df66-6e58-4c2e-af19-90afa4dc7441 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:21.271321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:21.272134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:21.272937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:21.272937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:21.272937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:21.273088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d2a5c-8003-4416-9604-d3f81fea1a5b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:21.275029 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 241/476] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:21.276024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3df66-6e58-4c2e-af19-90afa4dc7441 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:21.294740 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:21.384370 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3df66-6e58-4c2e-af19-90afa4dc7441 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:21.401791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d2a5c-8003-4416-9604-d3f81fea1a5b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:21.436227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20480ee4-befc-49c4-991d-8811a2ada482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:21.436973 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20480ee4-befc-49c4-991d-8811a2ada482 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:21.437468 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20480ee4-befc-49c4-991d-8811a2ada482 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:21.439196 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20480ee4-befc-49c4-991d-8811a2ada482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:21.439608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:21.451356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20480ee4-befc-49c4-991d-8811a2ada482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:21.452272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81f490-0cee-4c28-aa9d-6e754a8cf3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:21.452821 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20480ee4-befc-49c4-991d-8811a2ada482 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:21.452925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81f490-0cee-4c28-aa9d-6e754a8cf3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:21.453164 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81f490-0cee-4c28-aa9d-6e754a8cf3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:21.454269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81f490-0cee-4c28-aa9d-6e754a8cf3c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:21.455097 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81f490-0cee-4c28-aa9d-6e754a8cf3c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:21.455419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81f490-0cee-4c28-aa9d-6e754a8cf3c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:21.456957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81f490-0cee-4c28-aa9d-6e754a8cf3c8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-20480ee4-befc-49c4-991d-8811a2ada482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:21.468210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:21.469218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:21.470155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:21.470155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:21.470155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:21.470155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 242/477] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:21.470378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20480ee4-befc-49c4-991d-8811a2ada482 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:21.475094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 236/478] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:21.475804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81f490-0cee-4c28-aa9d-6e754a8cf3c8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:21.545916 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:21.581832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20480ee4-befc-49c4-991d-8811a2ada482 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:21.588110 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81f490-0cee-4c28-aa9d-6e754a8cf3c8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:21.616561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425846fc-736e-4799-ad35-3d42b2e38f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:21.617254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425846fc-736e-4799-ad35-3d42b2e38f73 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:21.617598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425846fc-736e-4799-ad35-3d42b2e38f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:21.617937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425846fc-736e-4799-ad35-3d42b2e38f73 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:21.618640 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425846fc-736e-4799-ad35-3d42b2e38f73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:21.619008 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425846fc-736e-4799-ad35-3d42b2e38f73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:21.620481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425846fc-736e-4799-ad35-3d42b2e38f73 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:21.626255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d14ef9-a616-45d5-96a4-1251bd4a6f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:21.626872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d14ef9-a616-45d5-96a4-1251bd4a6f93 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:21.627184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d14ef9-a616-45d5-96a4-1251bd4a6f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:21.627463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d14ef9-a616-45d5-96a4-1251bd4a6f93 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:21.628208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d14ef9-a616-45d5-96a4-1251bd4a6f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:21.628518 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d14ef9-a616-45d5-96a4-1251bd4a6f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:21.629944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d14ef9-a616-45d5-96a4-1251bd4a6f93 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:21.633505 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 243/479] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:21.634333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425846fc-736e-4799-ad35-3d42b2e38f73 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:21.644030 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 237/480] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:21.645027 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d14ef9-a616-45d5-96a4-1251bd4a6f93 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:21.704032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:21.736046 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425846fc-736e-4799-ad35-3d42b2e38f73 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:21.763376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:21.768303 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d14ef9-a616-45d5-96a4-1251bd4a6f93 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:21.771701 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fd929-49a1-4642-be61-921372d03da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:21.772224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fd929-49a1-4642-be61-921372d03da8 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:21.772500 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fd929-49a1-4642-be61-921372d03da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:21.772769 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fd929-49a1-4642-be61-921372d03da8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:21.773496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fd929-49a1-4642-be61-921372d03da8 None admin] RBAC: Adding query filter params (role.id=8adf2afafb0446a6992e9aeb380f645d, scope.project.id=f28d337589024cc88184355bd0ee8572, user.id=8c05a1b303c2418fb2941b8ef56f8972) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:21.773804 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fd929-49a1-4642-be61-921372d03da8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:21.775255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fd929-49a1-4642-be61-921372d03da8 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:21.782121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 244/481] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:12:21 2022] GET /identity/v3/role_assignments?user.id=8c05a1b303c2418fb2941b8ef56f8972&scope.project.id=f28d337589024cc88184355bd0ee8572&role.id=8adf2afafb0446a6992e9aeb380f645d => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:21.785144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fd929-49a1-4642-be61-921372d03da8 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:21.814755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2499afa-cc4d-4f03-9af3-bb426d7e6038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:21.815771 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2499afa-cc4d-4f03-9af3-bb426d7e6038 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:21.816272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2499afa-cc4d-4f03-9af3-bb426d7e6038 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:21.816687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2499afa-cc4d-4f03-9af3-bb426d7e6038 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:21.827155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2499afa-cc4d-4f03-9af3-bb426d7e6038 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:21.830649 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2499afa-cc4d-4f03-9af3-bb426d7e6038 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-a2499afa-cc4d-4f03-9af3-bb426d7e6038 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:21.843629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:21.845840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:21.846830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:21.846830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:21.846830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:21.846830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:21.846830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:21.846830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:21.846830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:21.846830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:21.853286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 238/482] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:21.854730 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2499afa-cc4d-4f03-9af3-bb426d7e6038 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:21.920798 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fd929-49a1-4642-be61-921372d03da8 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:21.957280 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3d405-562f-4e87-ac67-ac71b46a39cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:21.958211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3d405-562f-4e87-ac67-ac71b46a39cd None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:21.958211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3d405-562f-4e87-ac67-ac71b46a39cd None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:21.958463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3d405-562f-4e87-ac67-ac71b46a39cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:21.966260 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3d405-562f-4e87-ac67-ac71b46a39cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:21.967642 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3d405-562f-4e87-ac67-ac71b46a39cd None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-26b3d405-562f-4e87-ac67-ac71b46a39cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:21.980941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:21.982012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:21.983013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:21.983013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:21.983013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:21.983013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:21.983013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:21.983013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:21.983013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:21.983013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:21.983013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 245/483] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:21.992527 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3d405-562f-4e87-ac67-ac71b46a39cd None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:21.992814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2499afa-cc4d-4f03-9af3-bb426d7e6038 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:22.023246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:22.067457 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1798152-e0ae-49c8-85f2-281c5727ba5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:22.068009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1798152-e0ae-49c8-85f2-281c5727ba5c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:22.068287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1798152-e0ae-49c8-85f2-281c5727ba5c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:22.068556 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1798152-e0ae-49c8-85f2-281c5727ba5c None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:22.069264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1798152-e0ae-49c8-85f2-281c5727ba5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:22.069569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1798152-e0ae-49c8-85f2-281c5727ba5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:22.076501 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1798152-e0ae-49c8-85f2-281c5727ba5c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:22.099426 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:22.103963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 239/484] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:22.110518 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1798152-e0ae-49c8-85f2-281c5727ba5c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:22.173196 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3d405-562f-4e87-ac67-ac71b46a39cd None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:22.222671 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe42449-5ef5-4fd8-8f71-d3f19751e96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:22.223342 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe42449-5ef5-4fd8-8f71-d3f19751e96f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:22.223625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe42449-5ef5-4fd8-8f71-d3f19751e96f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:22.223898 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe42449-5ef5-4fd8-8f71-d3f19751e96f None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:22.224667 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe42449-5ef5-4fd8-8f71-d3f19751e96f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:22.225396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe42449-5ef5-4fd8-8f71-d3f19751e96f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:22.227161 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe42449-5ef5-4fd8-8f71-d3f19751e96f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:22.250695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 246/485] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:22.254331 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe42449-5ef5-4fd8-8f71-d3f19751e96f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:22.261443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1798152-e0ae-49c8-85f2-281c5727ba5c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:22.302872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245bc95-1742-44ff-8670-61d501b5afeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:22.303779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245bc95-1742-44ff-8670-61d501b5afeb None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:22.304334 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245bc95-1742-44ff-8670-61d501b5afeb None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:22.304784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245bc95-1742-44ff-8670-61d501b5afeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:22.305756 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245bc95-1742-44ff-8670-61d501b5afeb None admin] RBAC: Adding query filter params (role.id=8adf2afafb0446a6992e9aeb380f645d, scope.project.id=f28d337589024cc88184355bd0ee8572, user.id=6ff451e43cea4bb0a5fd73491a05bf82) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:22.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245bc95-1742-44ff-8670-61d501b5afeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:22.307957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245bc95-1742-44ff-8670-61d501b5afeb None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:22.315222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 240/486] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:12:22 2022] GET /identity/v3/role_assignments?user.id=6ff451e43cea4bb0a5fd73491a05bf82&scope.project.id=f28d337589024cc88184355bd0ee8572&role.id=8adf2afafb0446a6992e9aeb380f645d => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:22.378380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe42449-5ef5-4fd8-8f71-d3f19751e96f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:22.383925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357ff90-5306-421d-a24f-2dc51a0b1abc None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:22.384527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357ff90-5306-421d-a24f-2dc51a0b1abc None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:22.384997 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357ff90-5306-421d-a24f-2dc51a0b1abc None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:22.386971 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 241/487] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:12:22.396704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf0b78-c7e9-4f68-bf8a-d948ac02d681 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:22.396996 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf0b78-c7e9-4f68-bf8a-d948ac02d681 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:22.397251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf0b78-c7e9-4f68-bf8a-d948ac02d681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:22.398640 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:22.434276 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-bfaf0b78-c7e9-4f68-bf8a-d948ac02d681 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:22.435525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bfaf0b78-c7e9-4f68-bf8a-d948ac02d681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:22.447051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f4644-91d5-4aed-8e96-3203ff104927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:22.447705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f4644-91d5-4aed-8e96-3203ff104927 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:22.447984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f4644-91d5-4aed-8e96-3203ff104927 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:22.448255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f4644-91d5-4aed-8e96-3203ff104927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:22.449001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f4644-91d5-4aed-8e96-3203ff104927 None admin] RBAC: Adding query filter params (role.id=8adf2afafb0446a6992e9aeb380f645d, scope.project.id=f28d337589024cc88184355bd0ee8572, user.id=eb7a43e284e1401688b2e202c67a5eb5) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:22.449307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f4644-91d5-4aed-8e96-3203ff104927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:22.450799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f4644-91d5-4aed-8e96-3203ff104927 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:22.465865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 247/488] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:12:22 2022] GET /identity/v3/role_assignments?user.id=eb7a43e284e1401688b2e202c67a5eb5&scope.project.id=f28d337589024cc88184355bd0ee8572&role.id=8adf2afafb0446a6992e9aeb380f645d => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:22.499689 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:22.564118 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 242/489] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:12:22.851226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f4644-91d5-4aed-8e96-3203ff104927 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:22.917477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8afee-ea50-4ea3-9348-0c18197cdaf1 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:22.917799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8afee-ea50-4ea3-9348-0c18197cdaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:22.918035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8afee-ea50-4ea3-9348-0c18197cdaf1 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:22.919525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 243/490] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:12:22.925627 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24b7c5-da21-4634-b0f6-de5c23d3da5c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:22.926152 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24b7c5-da21-4634-b0f6-de5c23d3da5c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:22.926634 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24b7c5-da21-4634-b0f6-de5c23d3da5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:22.952632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ea24b7c5-da21-4634-b0f6-de5c23d3da5c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:22.955342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ea24b7c5-da21-4634-b0f6-de5c23d3da5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:23.027771 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f4644-91d5-4aed-8e96-3203ff104927 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:23.096002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 244/491] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:23.108576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554d902-bdf4-40d5-8897-40c1950759d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:23.109865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554d902-bdf4-40d5-8897-40c1950759d2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:23.110355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554d902-bdf4-40d5-8897-40c1950759d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:23.110810 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554d902-bdf4-40d5-8897-40c1950759d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:23.131894 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554d902-bdf4-40d5-8897-40c1950759d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-9554d902-bdf4-40d5-8897-40c1950759d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:23.137424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:23.138252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:23.139006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:23.139006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:23.139006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:23.139006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:23.139006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:23.139006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:23.139006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:23.139006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:23.139006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:23.139006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:23.139006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:23.139006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:23.141459 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 248/492] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:12:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:23.146893 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24b7c5-da21-4634-b0f6-de5c23d3da5c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:23.247970 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:23.256320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24b7c5-da21-4634-b0f6-de5c23d3da5c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:23.292295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f78efd-46b3-4aa9-931a-64eae1250a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:23.293667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f78efd-46b3-4aa9-931a-64eae1250a44 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:23.294137 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f78efd-46b3-4aa9-931a-64eae1250a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:23.294613 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f78efd-46b3-4aa9-931a-64eae1250a44 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:23.295497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f78efd-46b3-4aa9-931a-64eae1250a44 None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:23.295988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f78efd-46b3-4aa9-931a-64eae1250a44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:23.297766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f78efd-46b3-4aa9-931a-64eae1250a44 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:23.306428 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 245/493] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Oct 14 02:12:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:23.312307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554d902-bdf4-40d5-8897-40c1950759d2 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:23.355916 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:23.442861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554d902-bdf4-40d5-8897-40c1950759d2 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:23.487286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdca767-7dd2-4dca-a932-094fb2baa45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:23.488324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdca767-7dd2-4dca-a932-094fb2baa45c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:23.488801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdca767-7dd2-4dca-a932-094fb2baa45c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:23.489261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdca767-7dd2-4dca-a932-094fb2baa45c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:23.491318 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f78efd-46b3-4aa9-931a-64eae1250a44 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:23.512992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdca767-7dd2-4dca-a932-094fb2baa45c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:23.514465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdca767-7dd2-4dca-a932-094fb2baa45c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-2fdca767-7dd2-4dca-a932-094fb2baa45c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:23.534380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:23.535845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:23.536867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:23.536867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:23.536867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:23.536867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 249/494] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:23.541332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdca767-7dd2-4dca-a932-094fb2baa45c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:23.637211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:23.653520 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdca767-7dd2-4dca-a932-094fb2baa45c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:23.696195 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64f246-3b11-408e-ba18-01d6b9273ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:23.697177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64f246-3b11-408e-ba18-01d6b9273ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:23.697646 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64f246-3b11-408e-ba18-01d6b9273ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:23.698124 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64f246-3b11-408e-ba18-01d6b9273ec7 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:23.698946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64f246-3b11-408e-ba18-01d6b9273ec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:23.699507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64f246-3b11-408e-ba18-01d6b9273ec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:23.701355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64f246-3b11-408e-ba18-01d6b9273ec7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:23.718047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 250/495] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:23.721802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f78efd-46b3-4aa9-931a-64eae1250a44 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:23.724235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64f246-3b11-408e-ba18-01d6b9273ec7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:23.781882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d95f6-9138-4521-8b97-c64767e269f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:23.782243 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d95f6-9138-4521-8b97-c64767e269f2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:23.782542 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d95f6-9138-4521-8b97-c64767e269f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:23.782883 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d95f6-9138-4521-8b97-c64767e269f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:23.806597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d95f6-9138-4521-8b97-c64767e269f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-4e5d95f6-9138-4521-8b97-c64767e269f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:23.812057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:23.813832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:23.814914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:23.814914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:23.814914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:23.814914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:23.814914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:23.814914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:23.814914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:23.814914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:23.814914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:23.814914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:23.814914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:12:23.814914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:23.814914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 246/496] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:12:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:23.823973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d95f6-9138-4521-8b97-c64767e269f2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:23.865545 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:23.882628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64f246-3b11-408e-ba18-01d6b9273ec7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:23.920579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c9e2e-c11b-4d42-9c25-cc489ed6501b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:23.921131 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c9e2e-c11b-4d42-9c25-cc489ed6501b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:23.921406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c9e2e-c11b-4d42-9c25-cc489ed6501b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:23.921674 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c9e2e-c11b-4d42-9c25-cc489ed6501b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:23.930542 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c9e2e-c11b-4d42-9c25-cc489ed6501b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:23.932018 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c9e2e-c11b-4d42-9c25-cc489ed6501b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-756c9e2e-c11b-4d42-9c25-cc489ed6501b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:23.948381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:23.950360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:23.951731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:23.951731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:23.951731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:23.951731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:23.951731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:23.951731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:23.951731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:23.951731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:23.951731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:12:23.951731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:23.951731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 251/497] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:12:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:23.955962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c9e2e-c11b-4d42-9c25-cc489ed6501b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:23.967789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:24.074016 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c9e2e-c11b-4d42-9c25-cc489ed6501b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:24.088390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d95f6-9138-4521-8b97-c64767e269f2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:24.113995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe929a-c9cd-4d3b-9c3d-7504414924d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:24.114635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe929a-c9cd-4d3b-9c3d-7504414924d3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:24.114868 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe929a-c9cd-4d3b-9c3d-7504414924d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:24.115161 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe929a-c9cd-4d3b-9c3d-7504414924d3 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:24.115900 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe929a-c9cd-4d3b-9c3d-7504414924d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:24.116253 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe929a-c9cd-4d3b-9c3d-7504414924d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:24.117685 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe929a-c9cd-4d3b-9c3d-7504414924d3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:24.138508 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 252/498] 10.222.0.18 () {56 vars in 1135 bytes} [Fri Oct 14 02:12:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:24.143905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe929a-c9cd-4d3b-9c3d-7504414924d3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:24.167098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c65ae1-d1da-4216-8aa6-2bd8921ff794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:24.167098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c65ae1-d1da-4216-8aa6-2bd8921ff794 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:24.167098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c65ae1-d1da-4216-8aa6-2bd8921ff794 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:24.167098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c65ae1-d1da-4216-8aa6-2bd8921ff794 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:24.167872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c65ae1-d1da-4216-8aa6-2bd8921ff794 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:24.167872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c65ae1-d1da-4216-8aa6-2bd8921ff794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:24.168405 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c65ae1-d1da-4216-8aa6-2bd8921ff794 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:24.185849 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 247/499] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:12:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:24.196189 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c65ae1-d1da-4216-8aa6-2bd8921ff794 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:24.229579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:24.270920 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe929a-c9cd-4d3b-9c3d-7504414924d3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:24.306650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26795721-8908-422b-91cb-8e073d785ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:24.307671 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26795721-8908-422b-91cb-8e073d785ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:24.308235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26795721-8908-422b-91cb-8e073d785ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:24.308713 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26795721-8908-422b-91cb-8e073d785ae3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:24.347052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:24.374844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c65ae1-d1da-4216-8aa6-2bd8921ff794 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:24.404683 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26795721-8908-422b-91cb-8e073d785ae3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:24.406622 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26795721-8908-422b-91cb-8e073d785ae3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:24.439859 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c1766-6473-4096-881f-72faf14297e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:24.440436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c1766-6473-4096-881f-72faf14297e1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:24.440712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c1766-6473-4096-881f-72faf14297e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:24.440984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c1766-6473-4096-881f-72faf14297e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:24.468935 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c1766-6473-4096-881f-72faf14297e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:24.470725 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c1766-6473-4096-881f-72faf14297e1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:24.496615 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-ab2c1766-6473-4096-881f-72faf14297e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:24.497233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:24.498888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:24.503429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:24.503429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:24.503429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:24.504250 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 248/500] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:24.511394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c1766-6473-4096-881f-72faf14297e1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:24.577351 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 253/501] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:12:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:12:24.587103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26795721-8908-422b-91cb-8e073d785ae3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:24.713415 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c1766-6473-4096-881f-72faf14297e1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:24.748188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:24.756383 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fabb8-c965-48da-a4ea-3fea53ac32a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:24.757007 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fabb8-c965-48da-a4ea-3fea53ac32a1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:24.757283 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fabb8-c965-48da-a4ea-3fea53ac32a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:24.757553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fabb8-c965-48da-a4ea-3fea53ac32a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:24.758193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9fabb8-c965-48da-a4ea-3fea53ac32a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:24.758491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9fabb8-c965-48da-a4ea-3fea53ac32a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:24.760079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9fabb8-c965-48da-a4ea-3fea53ac32a1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:24.772635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 249/502] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:24.774075 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba657c0b-fad4-41a4-880f-55d1c14e1d9e None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:24.774356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba657c0b-fad4-41a4-880f-55d1c14e1d9e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:24.774623 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba657c0b-fad4-41a4-880f-55d1c14e1d9e None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:24.776670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 250/503] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:24.782053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba657c0b-fad4-41a4-880f-55d1c14e1d9e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:24.787517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26795721-8908-422b-91cb-8e073d785ae3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:24.830582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff22eb5-8610-4ad9-86fe-2c1b878997fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:24.831677 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff22eb5-8610-4ad9-86fe-2c1b878997fa None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:24.832229 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff22eb5-8610-4ad9-86fe-2c1b878997fa None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:24.832697 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff22eb5-8610-4ad9-86fe-2c1b878997fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:24.845425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff22eb5-8610-4ad9-86fe-2c1b878997fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:24.847349 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff22eb5-8610-4ad9-86fe-2c1b878997fa None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-4ff22eb5-8610-4ad9-86fe-2c1b878997fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:24.865180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:24.866258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:24.867171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:24.867171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:24.867171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:24.869354 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 254/504] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:24.871267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cae99-ac2c-4122-a4b7-184437250fff None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:24.871767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cae99-ac2c-4122-a4b7-184437250fff None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:24.872259 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cae99-ac2c-4122-a4b7-184437250fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:24.892716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-561cae99-ac2c-4122-a4b7-184437250fff None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:24.894311 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-561cae99-ac2c-4122-a4b7-184437250fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:24.910678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:24.925195 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba657c0b-fad4-41a4-880f-55d1c14e1d9e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:24.973124 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 255/505] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:24.974836 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cae99-ac2c-4122-a4b7-184437250fff None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:24.983861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea914c5-bacb-42ad-b033-73a47a1be79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:24.985651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea914c5-bacb-42ad-b033-73a47a1be79b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:24.987021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea914c5-bacb-42ad-b033-73a47a1be79b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:24.987554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea914c5-bacb-42ad-b033-73a47a1be79b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:25.004025 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea914c5-bacb-42ad-b033-73a47a1be79b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:25.007084 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea914c5-bacb-42ad-b033-73a47a1be79b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-cea914c5-bacb-42ad-b033-73a47a1be79b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:25.033877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:25.034958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:25.035916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:25.035916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:25.035916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:25.035916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:25.035916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:25.035916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:25.035916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:25.035916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:25.035916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:12:25.035916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:25.040975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 251/506] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Oct 14 02:12:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:25.045884 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea914c5-bacb-42ad-b033-73a47a1be79b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:25.172603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:25.176435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cae99-ac2c-4122-a4b7-184437250fff None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:25.247951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd50a2-784e-4a1c-b6c1-5b4d8383ab99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:25.248749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd50a2-784e-4a1c-b6c1-5b4d8383ab99 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:25.249126 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd50a2-784e-4a1c-b6c1-5b4d8383ab99 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:25.249582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd50a2-784e-4a1c-b6c1-5b4d8383ab99 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:25.250913 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd50a2-784e-4a1c-b6c1-5b4d8383ab99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:25.251314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd50a2-784e-4a1c-b6c1-5b4d8383ab99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:25.252608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd50a2-784e-4a1c-b6c1-5b4d8383ab99 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:25.265043 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 256/507] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:25.268422 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2c70e-6689-426d-b44c-3d01d0bbdc35 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:25.268710 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2c70e-6689-426d-b44c-3d01d0bbdc35 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:25.268981 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2c70e-6689-426d-b44c-3d01d0bbdc35 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:25.270330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 257/508] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:25.274685 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8299707-10ce-48d2-8ae3-815e79404a3d None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:25.274973 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8299707-10ce-48d2-8ae3-815e79404a3d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:25.275286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8299707-10ce-48d2-8ae3-815e79404a3d None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:25.277127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 258/509] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:25.279842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea914c5-bacb-42ad-b033-73a47a1be79b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:25.280264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8299707-10ce-48d2-8ae3-815e79404a3d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:25.294535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:25.323907 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8b668-a6f6-4afb-a738-541aa4d1684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:25.324565 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8b668-a6f6-4afb-a738-541aa4d1684c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:25.324853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8b668-a6f6-4afb-a738-541aa4d1684c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:25.325125 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8b668-a6f6-4afb-a738-541aa4d1684c None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:25.325880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8b668-a6f6-4afb-a738-541aa4d1684c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:25.326186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8b668-a6f6-4afb-a738-541aa4d1684c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:25.327701 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8b668-a6f6-4afb-a738-541aa4d1684c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:25.345992 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 252/510] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Oct 14 02:12:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:25.347290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac2428-5e55-43bc-85ec-b37f0e59a390 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:25.347867 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac2428-5e55-43bc-85ec-b37f0e59a390 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:25.347867 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac2428-5e55-43bc-85ec-b37f0e59a390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:25.366753 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-61ac2428-5e55-43bc-85ec-b37f0e59a390 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:25.368028 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-61ac2428-5e55-43bc-85ec-b37f0e59a390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:25.401747 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8299707-10ce-48d2-8ae3-815e79404a3d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:25.439854 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180d329-510d-48d4-95b1-e6cf4ccb4cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:25.440480 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180d329-510d-48d4-95b1-e6cf4ccb4cbe None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:25.440754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180d329-510d-48d4-95b1-e6cf4ccb4cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:25.441023 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180d329-510d-48d4-95b1-e6cf4ccb4cbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:25.449283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180d329-510d-48d4-95b1-e6cf4ccb4cbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:25.450462 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 253/511] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:25.451813 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180d329-510d-48d4-95b1-e6cf4ccb4cbe None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:25.451920 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349520c-2388-4eac-9f1e-ec9f9594de7b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:25.452108 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349520c-2388-4eac-9f1e-ec9f9594de7b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:25.452390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349520c-2388-4eac-9f1e-ec9f9594de7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a180d329-510d-48d4-95b1-e6cf4ccb4cbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:25.474448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:25.481060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:25.482146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:25.482146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:25.482146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:25.482146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:25.482146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:25.482146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:25.482146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:25.482146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:25.482653 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5349520c-2388-4eac-9f1e-ec9f9594de7b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:25.482799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 259/512] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:25.482799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180d329-510d-48d4-95b1-e6cf4ccb4cbe None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:25.483059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5349520c-2388-4eac-9f1e-ec9f9594de7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:25.524443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:25.575111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 254/513] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:25.575111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349520c-2388-4eac-9f1e-ec9f9594de7b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:25.575939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:25.612360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180d329-510d-48d4-95b1-e6cf4ccb4cbe None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:25.647138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb024a-0af6-4819-8472-4eaca565223c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:25.647999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb024a-0af6-4819-8472-4eaca565223c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:25.648444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb024a-0af6-4819-8472-4eaca565223c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:25.648855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb024a-0af6-4819-8472-4eaca565223c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:25.660182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb024a-0af6-4819-8472-4eaca565223c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:25.661834 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb024a-0af6-4819-8472-4eaca565223c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-53cb024a-0af6-4819-8472-4eaca565223c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:25.676874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:25.677820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:25.679346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:25.679346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:25.679346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:25.679346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 260/514] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:25.679346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb024a-0af6-4819-8472-4eaca565223c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:25.700565 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349520c-2388-4eac-9f1e-ec9f9594de7b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:25.746400 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d94491-38dd-4e5f-b100-596eecd3ac11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:25.747042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d94491-38dd-4e5f-b100-596eecd3ac11 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:25.747612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d94491-38dd-4e5f-b100-596eecd3ac11 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:25.747881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d94491-38dd-4e5f-b100-596eecd3ac11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:25.758158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d94491-38dd-4e5f-b100-596eecd3ac11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:25.759633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d94491-38dd-4e5f-b100-596eecd3ac11 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-04d94491-38dd-4e5f-b100-596eecd3ac11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:25.774880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:25.776070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:25.776962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:25.776962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:25.776962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:25.776962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 255/515] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:25.776962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d94491-38dd-4e5f-b100-596eecd3ac11 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:25.796652 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb024a-0af6-4819-8472-4eaca565223c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:25.808418 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:25.817568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:25.833052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cede91-14dc-48e8-bee3-f06ae9f1cd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:25.833657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cede91-14dc-48e8-bee3-f06ae9f1cd9f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:25.834207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cede91-14dc-48e8-bee3-f06ae9f1cd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:25.834482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cede91-14dc-48e8-bee3-f06ae9f1cd9f None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:25.835193 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cede91-14dc-48e8-bee3-f06ae9f1cd9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:25.835497 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cede91-14dc-48e8-bee3-f06ae9f1cd9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:25.836917 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cede91-14dc-48e8-bee3-f06ae9f1cd9f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:25.850122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 261/516] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:25.851111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cede91-14dc-48e8-bee3-f06ae9f1cd9f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:25.898812 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d94491-38dd-4e5f-b100-596eecd3ac11 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:25.932889 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3f070-d56c-459d-9c42-ed4f8a0bbb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:25.933504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3f070-d56c-459d-9c42-ed4f8a0bbb22 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:25.933776 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3f070-d56c-459d-9c42-ed4f8a0bbb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:25.934046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3f070-d56c-459d-9c42-ed4f8a0bbb22 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:25.934679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3f070-d56c-459d-9c42-ed4f8a0bbb22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:25.934982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3f070-d56c-459d-9c42-ed4f8a0bbb22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:25.936364 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3f070-d56c-459d-9c42-ed4f8a0bbb22 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:25.947748 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 256/517] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:25.948588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3f070-d56c-459d-9c42-ed4f8a0bbb22 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:25.950181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cede91-14dc-48e8-bee3-f06ae9f1cd9f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:25.984044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e4831-4cf7-46c7-8a4e-9706a2fc337c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:25.984696 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e4831-4cf7-46c7-8a4e-9706a2fc337c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:25.984979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e4831-4cf7-46c7-8a4e-9706a2fc337c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:25.985249 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e4831-4cf7-46c7-8a4e-9706a2fc337c None admin] PATH_INFO: `/v3/services/network` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:25.985825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e4831-4cf7-46c7-8a4e-9706a2fc337c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:25.987214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e4831-4cf7-46c7-8a4e-9706a2fc337c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-ec5e4831-4cf7-46c7-8a4e-9706a2fc337c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:25.997763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:25.998776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:25.999644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:25.999644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:25.999644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:25.999644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:25.999644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:25.999644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:25.999644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:25.999644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:25.999644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:25.999644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:25.999644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:25.999644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:25.999644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:12:25.999644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:26.003942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 262/518] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:25 2022] GET /identity/v3/services/network => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:26.004071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e4831-4cf7-46c7-8a4e-9706a2fc337c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:26.029143 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:26.047986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3f070-d56c-459d-9c42-ed4f8a0bbb22 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:26.048821 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:26.081632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcdb26-5b47-4f08-9592-491fe4f22fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:26.082247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcdb26-5b47-4f08-9592-491fe4f22fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:26.082527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcdb26-5b47-4f08-9592-491fe4f22fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:26.082802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcdb26-5b47-4f08-9592-491fe4f22fd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:26.083485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcdb26-5b47-4f08-9592-491fe4f22fd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:26.083793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcdb26-5b47-4f08-9592-491fe4f22fd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:26.085169 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcdb26-5b47-4f08-9592-491fe4f22fd6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:26.096845 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 257/519] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:26.097629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcdb26-5b47-4f08-9592-491fe4f22fd6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:26.101767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e4831-4cf7-46c7-8a4e-9706a2fc337c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:26.134743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903f365-bfc4-4dc9-bc22-756a85186cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:26.135490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903f365-bfc4-4dc9-bc22-756a85186cfc None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:26.135770 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903f365-bfc4-4dc9-bc22-756a85186cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:26.136043 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903f365-bfc4-4dc9-bc22-756a85186cfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:26.136763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903f365-bfc4-4dc9-bc22-756a85186cfc None admin] RBAC: Adding query filter params (role.id=8adf2afafb0446a6992e9aeb380f645d, scope.project.id=f28d337589024cc88184355bd0ee8572, user.id=cf0efd6d12ad458388e0c3574bbedc6a) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:26.137070 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903f365-bfc4-4dc9-bc22-756a85186cfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:26.138518 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903f365-bfc4-4dc9-bc22-756a85186cfc None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:26.144918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 263/520] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:12:26 2022] GET /identity/v3/role_assignments?user.id=cf0efd6d12ad458388e0c3574bbedc6a&scope.project.id=f28d337589024cc88184355bd0ee8572&role.id=8adf2afafb0446a6992e9aeb380f645d => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:26.145752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903f365-bfc4-4dc9-bc22-756a85186cfc None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:26.212081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcdb26-5b47-4f08-9592-491fe4f22fd6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:26.251404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1fb8f-0a01-4eed-9c83-753e62b9f764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:26.252157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1fb8f-0a01-4eed-9c83-753e62b9f764 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:26.252502 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1fb8f-0a01-4eed-9c83-753e62b9f764 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:26.252833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1fb8f-0a01-4eed-9c83-753e62b9f764 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:26.253492 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1fb8f-0a01-4eed-9c83-753e62b9f764 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:26.254966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1fb8f-0a01-4eed-9c83-753e62b9f764 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:26.256381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903f365-bfc4-4dc9-bc22-756a85186cfc None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:26.266448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-bdf1fb8f-0a01-4eed-9c83-753e62b9f764 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:26.267768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:26.268910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:26.269903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:26.269903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:26.269903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:26.269903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:26.269903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:26.269903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:26.269903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:26.269903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:26.269903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:26.269903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:26.269903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:26.269903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:26.269903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:12:26.269903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:26.271266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 258/521] 10.222.0.18 () {56 vars in 1129 bytes} [Fri Oct 14 02:12:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:26.272228 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1fb8f-0a01-4eed-9c83-753e62b9f764 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:26.281813 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:26.302876 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01106c-de60-410a-a70f-9e980d8f4b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:26.303533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01106c-de60-410a-a70f-9e980d8f4b9d None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:26.303811 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01106c-de60-410a-a70f-9e980d8f4b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:26.304114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01106c-de60-410a-a70f-9e980d8f4b9d None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:26.304740 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01106c-de60-410a-a70f-9e980d8f4b9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:26.306201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01106c-de60-410a-a70f-9e980d8f4b9d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:26.330631 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 264/522] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:12:26 2022] POST /identity/v3/users => generated 254 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:26.331496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01106c-de60-410a-a70f-9e980d8f4b9d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:26.396742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1fb8f-0a01-4eed-9c83-753e62b9f764 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:26.450510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57707ba-bda7-46e9-a880-2600adf34004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:26.451800 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57707ba-bda7-46e9-a880-2600adf34004 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:26.452197 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57707ba-bda7-46e9-a880-2600adf34004 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:26.452475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57707ba-bda7-46e9-a880-2600adf34004 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:26.453651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57707ba-bda7-46e9-a880-2600adf34004 None admin] RBAC: Adding query filter params (name=network) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:26.453968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57707ba-bda7-46e9-a880-2600adf34004 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:26.456299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57707ba-bda7-46e9-a880-2600adf34004 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:26.460911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01106c-de60-410a-a70f-9e980d8f4b9d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:26.462645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 259/523] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Oct 14 02:12:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:26.463739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57707ba-bda7-46e9-a880-2600adf34004 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:26.518271 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab98b1-b319-4b48-b351-0e8d9ca7e601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:26.519233 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab98b1-b319-4b48-b351-0e8d9ca7e601 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:26.519518 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab98b1-b319-4b48-b351-0e8d9ca7e601 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:26.519933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab98b1-b319-4b48-b351-0e8d9ca7e601 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:26.528603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab98b1-b319-4b48-b351-0e8d9ca7e601 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:26.531055 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab98b1-b319-4b48-b351-0e8d9ca7e601 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-5eab98b1-b319-4b48-b351-0e8d9ca7e601 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:26.547524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:26.548861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:26.549725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:26.549725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:26.549725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:26.549725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:26.549725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:26.549725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:26.549725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:26.549725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:26.549725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 265/524] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:26.550676 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab98b1-b319-4b48-b351-0e8d9ca7e601 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:26.556984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:26.583698 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:26.593420 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57707ba-bda7-46e9-a880-2600adf34004 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:26.633507 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6761d-4591-4e33-a60e-bf50ad338afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:26.634591 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6761d-4591-4e33-a60e-bf50ad338afc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:26.635127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6761d-4591-4e33-a60e-bf50ad338afc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:26.635708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6761d-4591-4e33-a60e-bf50ad338afc None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:26.636553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6761d-4591-4e33-a60e-bf50ad338afc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:26.637058 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6761d-4591-4e33-a60e-bf50ad338afc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:26.638933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6761d-4591-4e33-a60e-bf50ad338afc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:26.647424 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 260/525] 10.222.0.18 () {56 vars in 1124 bytes} [Fri Oct 14 02:12:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:26.649077 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6761d-4591-4e33-a60e-bf50ad338afc None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:26.673876 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab98b1-b319-4b48-b351-0e8d9ca7e601 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:26.718123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411455b-f383-43f9-932b-941a49feae88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:26.719032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411455b-f383-43f9-932b-941a49feae88 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:26.719445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411455b-f383-43f9-932b-941a49feae88 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:26.720149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411455b-f383-43f9-932b-941a49feae88 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:26.720814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411455b-f383-43f9-932b-941a49feae88 None admin] RBAC: Adding query filter params (type=network) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:26.721104 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411455b-f383-43f9-932b-941a49feae88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:26.722984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411455b-f383-43f9-932b-941a49feae88 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:26.739118 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 266/526] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Oct 14 02:12:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:26.739118 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411455b-f383-43f9-932b-941a49feae88 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:26.831171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6761d-4591-4e33-a60e-bf50ad338afc None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:26.879572 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:26.887757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63e699-4be2-4c81-a745-4e3b1f655da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:26.888047 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63e699-4be2-4c81-a745-4e3b1f655da5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:26.891109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63e699-4be2-4c81-a745-4e3b1f655da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:26.891109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63e699-4be2-4c81-a745-4e3b1f655da5 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:26.891109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63e699-4be2-4c81-a745-4e3b1f655da5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:26.891109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63e699-4be2-4c81-a745-4e3b1f655da5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:26.891489 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63e699-4be2-4c81-a745-4e3b1f655da5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:26.893063 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:26.905516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 261/527] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:26.909037 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411455b-f383-43f9-932b-941a49feae88 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:26.912022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63e699-4be2-4c81-a745-4e3b1f655da5 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:26.977090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c178250-35e3-405a-a34c-bd31ba6b13dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:26.978275 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c178250-35e3-405a-a34c-bd31ba6b13dc None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:26.978730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c178250-35e3-405a-a34c-bd31ba6b13dc None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:26.979302 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c178250-35e3-405a-a34c-bd31ba6b13dc None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:26.980416 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c178250-35e3-405a-a34c-bd31ba6b13dc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:26.981138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c178250-35e3-405a-a34c-bd31ba6b13dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:26.983158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c178250-35e3-405a-a34c-bd31ba6b13dc None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:26.995285 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 267/528] 10.222.0.18 () {56 vars in 1124 bytes} [Fri Oct 14 02:12:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:27.055570 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63e699-4be2-4c81-a745-4e3b1f655da5 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:27.112444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb2611-9345-4a1c-8f38-02baf4d06dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:27.113109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb2611-9345-4a1c-8f38-02baf4d06dc0 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:27.113387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb2611-9345-4a1c-8f38-02baf4d06dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:27.113661 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb2611-9345-4a1c-8f38-02baf4d06dc0 None admin] PATH_INFO: `/v3/projects/f28d337589024cc88184355bd0ee8572/users/de13be55b4034477baf4c029c1a79757/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:27.120349 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb2611-9345-4a1c-8f38-02baf4d06dc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f28d337589024cc88184355bd0ee8572, user_id=de13be55b4034477baf4c029c1a79757, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:27.121825 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb2611-9345-4a1c-8f38-02baf4d06dc0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:27.151406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 262/529] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:12:26 2022] PUT /identity/v3/projects/f28d337589024cc88184355bd0ee8572/users/de13be55b4034477baf4c029c1a79757/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:12:28.815569 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c89953-7364-4ff0-8f9b-3431b292c749 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:28.816072 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c89953-7364-4ff0-8f9b-3431b292c749 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:28.816146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c89953-7364-4ff0-8f9b-3431b292c749 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:28.817694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 268/530] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:28.822968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f564d98-05e4-480b-b007-bc4f7bbd94ec None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:28.823385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f564d98-05e4-480b-b007-bc4f7bbd94ec None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:28.823549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f564d98-05e4-480b-b007-bc4f7bbd94ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:28.852065 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-8f564d98-05e4-480b-b007-bc4f7bbd94ec None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:28.853266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8f564d98-05e4-480b-b007-bc4f7bbd94ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:28.887091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e2fe2-5b06-40d8-9de2-285c241d9846 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:28.887091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e2fe2-5b06-40d8-9de2-285c241d9846 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:28.887091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e2fe2-5b06-40d8-9de2-285c241d9846 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:28.888960 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 269/531] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:28.897267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a40a8d-fd32-483a-8f70-c2f348087be1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:28.897824 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a40a8d-fd32-483a-8f70-c2f348087be1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:28.898297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a40a8d-fd32-483a-8f70-c2f348087be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:28.932523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e9a40a8d-fd32-483a-8f70-c2f348087be1 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:28.934883 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e9a40a8d-fd32-483a-8f70-c2f348087be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:28.943412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:28.972046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 263/532] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:29.048739 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 270/533] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:29.327628 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f564d98-05e4-480b-b007-bc4f7bbd94ec None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:29.438814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f564d98-05e4-480b-b007-bc4f7bbd94ec None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:29.461427 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a40a8d-fd32-483a-8f70-c2f348087be1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:29.478308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ebf0b-9c6d-47df-b401-65dbc68ed348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:29.478981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ebf0b-9c6d-47df-b401-65dbc68ed348 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:29.479299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ebf0b-9c6d-47df-b401-65dbc68ed348 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:29.479578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ebf0b-9c6d-47df-b401-65dbc68ed348 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:29.493259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ebf0b-9c6d-47df-b401-65dbc68ed348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-6e5ebf0b-9c6d-47df-b401-65dbc68ed348 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:29.498463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:29.499452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:29.500686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:29.500686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:29.500686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:29.500686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:29.500686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:29.500686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:29.500686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:29.500686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:29.500686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:29.500686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:29.500686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:12:29.500686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:29.503999 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 264/534] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:12:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:29.508061 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ebf0b-9c6d-47df-b401-65dbc68ed348 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:29.587562 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:29.627842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ebf0b-9c6d-47df-b401-65dbc68ed348 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:29.648761 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:29.668801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a40a8d-fd32-483a-8f70-c2f348087be1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:29.673229 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66801d6c-d1b1-4931-beee-03bae7008967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:29.674043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66801d6c-d1b1-4931-beee-03bae7008967 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:29.674472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66801d6c-d1b1-4931-beee-03bae7008967 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:29.674917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66801d6c-d1b1-4931-beee-03bae7008967 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:29.675790 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66801d6c-d1b1-4931-beee-03bae7008967 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:29.676237 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66801d6c-d1b1-4931-beee-03bae7008967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:29.677765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66801d6c-d1b1-4931-beee-03bae7008967 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:29.694202 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 265/535] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:12:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:29.696173 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603edd9c-60c8-42f9-8503-924579734131 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:29.696619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603edd9c-60c8-42f9-8503-924579734131 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:29.697035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603edd9c-60c8-42f9-8503-924579734131 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:29.698923 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 266/536] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:29.699880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603edd9c-60c8-42f9-8503-924579734131 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:29.709784 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c4eb3-3b10-4af3-bd0a-ef8b38924800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:29.710415 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c4eb3-3b10-4af3-bd0a-ef8b38924800 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:29.710687 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c4eb3-3b10-4af3-bd0a-ef8b38924800 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:29.710955 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c4eb3-3b10-4af3-bd0a-ef8b38924800 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:29.721595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c4eb3-3b10-4af3-bd0a-ef8b38924800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-b34c4eb3-3b10-4af3-bd0a-ef8b38924800 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:29.729266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:29.730157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:29.730971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:29.730971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:29.730971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:29.730971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:29.730971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:29.730971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:29.730971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:29.730971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:29.730971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:29.730971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:29.730971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:29.730971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:29.735727 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 271/537] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:12:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:29.737764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42274110-2bd0-4fbf-bafa-78ad4e59563a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:29.738694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42274110-2bd0-4fbf-bafa-78ad4e59563a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:29.739490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42274110-2bd0-4fbf-bafa-78ad4e59563a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:29.762298 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-42274110-2bd0-4fbf-bafa-78ad4e59563a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:29.763695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-42274110-2bd0-4fbf-bafa-78ad4e59563a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:29.822911 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603edd9c-60c8-42f9-8503-924579734131 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:29.831467 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:29.837358 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 272/538] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:29.838754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee789e92-da00-4e3d-b698-7b9018231b9c None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:29.838990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee789e92-da00-4e3d-b698-7b9018231b9c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:29.839288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee789e92-da00-4e3d-b698-7b9018231b9c None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:29.844552 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 273/539] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:29.844552 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee789e92-da00-4e3d-b698-7b9018231b9c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:29.881350 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69cc21b-c9bb-401f-9a07-739c0728c885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:29.882336 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cc21b-c9bb-401f-9a07-739c0728c885 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:29.883111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cc21b-c9bb-401f-9a07-739c0728c885 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:29.883537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cc21b-c9bb-401f-9a07-739c0728c885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:29.899123 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cc21b-c9bb-401f-9a07-739c0728c885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:29.901178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69cc21b-c9bb-401f-9a07-739c0728c885 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-f69cc21b-c9bb-401f-9a07-739c0728c885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:29.918555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:29.919754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:29.920734 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:29.920734 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:29.920734 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:29.923433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 267/540] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:29.925334 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:29.926464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a063ac-6fdd-4f38-8390-76b495cf64f4 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:29.926759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a063ac-6fdd-4f38-8390-76b495cf64f4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:29.927128 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a063ac-6fdd-4f38-8390-76b495cf64f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:29.948164 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d6a063ac-6fdd-4f38-8390-76b495cf64f4 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:29.949545 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d6a063ac-6fdd-4f38-8390-76b495cf64f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:29.956796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee789e92-da00-4e3d-b698-7b9018231b9c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:29.992960 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82777355-eaac-420e-8156-40b90b18037e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:29.993571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82777355-eaac-420e-8156-40b90b18037e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:29.993850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82777355-eaac-420e-8156-40b90b18037e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:29.994124 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82777355-eaac-420e-8156-40b90b18037e None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:29.994792 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82777355-eaac-420e-8156-40b90b18037e None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:29.995123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82777355-eaac-420e-8156-40b90b18037e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:29.996498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82777355-eaac-420e-8156-40b90b18037e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:30.005307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 274/541] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Oct 14 02:12:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:30.006474 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51048ed3-65d1-413e-9ad4-fb0c22b644c8 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:30.006762 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51048ed3-65d1-413e-9ad4-fb0c22b644c8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:30.007039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51048ed3-65d1-413e-9ad4-fb0c22b644c8 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:30.008332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 275/542] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:30.009150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51048ed3-65d1-413e-9ad4-fb0c22b644c8 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:30.031607 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 268/543] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:30.032462 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a063ac-6fdd-4f38-8390-76b495cf64f4 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:30.120235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51048ed3-65d1-413e-9ad4-fb0c22b644c8 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:30.139083 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:30.154244 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a063ac-6fdd-4f38-8390-76b495cf64f4 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:30.156101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54a351-49e3-43d1-9ed1-f7993f1f44ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:30.156811 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54a351-49e3-43d1-9ed1-f7993f1f44ff None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:30.157088 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54a351-49e3-43d1-9ed1-f7993f1f44ff None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:30.157353 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54a351-49e3-43d1-9ed1-f7993f1f44ff None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:30.157986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54a351-49e3-43d1-9ed1-f7993f1f44ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:30.158282 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54a351-49e3-43d1-9ed1-f7993f1f44ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:30.159659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54a351-49e3-43d1-9ed1-f7993f1f44ff None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:30.161243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:30.176504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 276/544] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:30.176504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b0868-5e30-48af-ac76-dbb74df31ebb None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:30.176504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b0868-5e30-48af-ac76-dbb74df31ebb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:30.176504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b0868-5e30-48af-ac76-dbb74df31ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:30.194024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-469b0868-5e30-48af-ac76-dbb74df31ebb None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:30.195238 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-469b0868-5e30-48af-ac76-dbb74df31ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:30.198770 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc29064-6575-4916-85e4-5a3b017f775a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:30.199816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc29064-6575-4916-85e4-5a3b017f775a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:30.200296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc29064-6575-4916-85e4-5a3b017f775a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:30.200816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc29064-6575-4916-85e4-5a3b017f775a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:30.213959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc29064-6575-4916-85e4-5a3b017f775a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:30.215704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc29064-6575-4916-85e4-5a3b017f775a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-9bc29064-6575-4916-85e4-5a3b017f775a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:30.234810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:30.236069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:30.236956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:30.236956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:30.236956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:30.236956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 269/545] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:30.237201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc29064-6575-4916-85e4-5a3b017f775a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:30.269865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 277/546] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:12:30.270813 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b0868-5e30-48af-ac76-dbb74df31ebb None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:30.360263 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc29064-6575-4916-85e4-5a3b017f775a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:30.377634 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b0868-5e30-48af-ac76-dbb74df31ebb None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:30.396941 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c323d2-6c1d-4a80-a32c-44e8ca9a8546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:30.397550 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c323d2-6c1d-4a80-a32c-44e8ca9a8546 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:30.397823 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c323d2-6c1d-4a80-a32c-44e8ca9a8546 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:30.398087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c323d2-6c1d-4a80-a32c-44e8ca9a8546 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:30.398728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c323d2-6c1d-4a80-a32c-44e8ca9a8546 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:30.400139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c323d2-6c1d-4a80-a32c-44e8ca9a8546 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:30.402906 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:30.413830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:30.419334 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807acd9-d00b-4dba-b0b4-4b406dfe896e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:30.419967 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807acd9-d00b-4dba-b0b4-4b406dfe896e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:30.420240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807acd9-d00b-4dba-b0b4-4b406dfe896e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:30.420510 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807acd9-d00b-4dba-b0b4-4b406dfe896e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:30.421332 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 270/547] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Oct 14 02:12:30 2022] POST /identity/v3/services => generated 240 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:30.422343 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c323d2-6c1d-4a80-a32c-44e8ca9a8546 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:30.428561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807acd9-d00b-4dba-b0b4-4b406dfe896e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:30.430027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807acd9-d00b-4dba-b0b4-4b406dfe896e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-7807acd9-d00b-4dba-b0b4-4b406dfe896e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:30.443093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:30.444264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:30.445224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:30.445224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:30.445224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:30.445224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:30.445224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:30.445224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:30.445224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:30.445224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:30.445224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:12:30.445224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:30.445224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 278/548] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:12:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:30.445918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807acd9-d00b-4dba-b0b4-4b406dfe896e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:30.574311 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807acd9-d00b-4dba-b0b4-4b406dfe896e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:30.590696 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c323d2-6c1d-4a80-a32c-44e8ca9a8546 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:30.618151 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634b887-ce0e-4065-bcda-8fdba8de7043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:30.618768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634b887-ce0e-4065-bcda-8fdba8de7043 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:30.619048 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634b887-ce0e-4065-bcda-8fdba8de7043 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:30.619350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634b887-ce0e-4065-bcda-8fdba8de7043 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:30.620012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634b887-ce0e-4065-bcda-8fdba8de7043 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:30.621413 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634b887-ce0e-4065-bcda-8fdba8de7043 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:30.633779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01a64e-4d37-4f37-9e6f-dc4e696de0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:30.634339 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01a64e-4d37-4f37-9e6f-dc4e696de0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:30.635280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01a64e-4d37-4f37-9e6f-dc4e696de0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:30.635280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01a64e-4d37-4f37-9e6f-dc4e696de0a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:30.635567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01a64e-4d37-4f37-9e6f-dc4e696de0a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:30.635878 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01a64e-4d37-4f37-9e6f-dc4e696de0a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:30.637264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01a64e-4d37-4f37-9e6f-dc4e696de0a9 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:30.639464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 279/549] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Oct 14 02:12:30 2022] POST /identity/v3/services => generated 251 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:30.640923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634b887-ce0e-4065-bcda-8fdba8de7043 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:30.649819 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 271/550] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:30.651118 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01a64e-4d37-4f37-9e6f-dc4e696de0a9 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:30.706374 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:30.734130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:30.794959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01a64e-4d37-4f37-9e6f-dc4e696de0a9 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:30.804093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634b887-ce0e-4065-bcda-8fdba8de7043 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:30.840859 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b942a-d259-4f3d-85d6-4190a85716aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:30.841518 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b942a-d259-4f3d-85d6-4190a85716aa None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:30.841803 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b942a-d259-4f3d-85d6-4190a85716aa None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:30.842107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b942a-d259-4f3d-85d6-4190a85716aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:30.851915 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f411046-5522-4db6-9297-99086f7b82f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:30.851915 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f411046-5522-4db6-9297-99086f7b82f3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:30.852478 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f411046-5522-4db6-9297-99086f7b82f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:30.852478 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f411046-5522-4db6-9297-99086f7b82f3 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:30.853161 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f411046-5522-4db6-9297-99086f7b82f3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:30.853462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f411046-5522-4db6-9297-99086f7b82f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:30.855118 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f411046-5522-4db6-9297-99086f7b82f3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:30.858465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b942a-d259-4f3d-85d6-4190a85716aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-a86b942a-d259-4f3d-85d6-4190a85716aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:30.863916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:30.864808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:30.865622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:30.865622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:30.865622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:30.865622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:30.865622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:30.865622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:30.865622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:30.865622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:30.865622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:30.865622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:30.865622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:12:30.865622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:30.865622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 272/551] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:12:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:30.866302 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b942a-d259-4f3d-85d6-4190a85716aa None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:30.878695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 280/552] 10.222.0.18 () {56 vars in 1135 bytes} [Fri Oct 14 02:12:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:30.879817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f411046-5522-4db6-9297-99086f7b82f3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:30.994040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:30.996158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b942a-d259-4f3d-85d6-4190a85716aa None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:30.998906 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f411046-5522-4db6-9297-99086f7b82f3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:31.017942 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:31.044606 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a892c69-ae31-4111-9206-73f109e026ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:31.045010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbab2ec-e93d-4a04-9e7d-cd7ecdfcf85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:31.045256 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a892c69-ae31-4111-9206-73f109e026ac None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:31.045537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a892c69-ae31-4111-9206-73f109e026ac None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:31.045813 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a892c69-ae31-4111-9206-73f109e026ac None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:31.046260 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbab2ec-e93d-4a04-9e7d-cd7ecdfcf85a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:31.047405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbab2ec-e93d-4a04-9e7d-cd7ecdfcf85a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:31.047896 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbab2ec-e93d-4a04-9e7d-cd7ecdfcf85a None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:31.048720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbab2ec-e93d-4a04-9e7d-cd7ecdfcf85a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:31.049169 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbab2ec-e93d-4a04-9e7d-cd7ecdfcf85a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:31.050702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbab2ec-e93d-4a04-9e7d-cd7ecdfcf85a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:31.055720 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a892c69-ae31-4111-9206-73f109e026ac None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:31.057770 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a892c69-ae31-4111-9206-73f109e026ac None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:31.065347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 281/553] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:12:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:31.066504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbab2ec-e93d-4a04-9e7d-cd7ecdfcf85a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-4a892c69-ae31-4111-9206-73f109e026ac None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:31.072029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:31.073151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:31.074201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:31.074201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:31.074201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:31.074201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:31.074201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:31.074201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:31.074201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:31.074201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:31.074201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:12:31.074201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:31.074201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 273/554] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Oct 14 02:12:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:31.074201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a892c69-ae31-4111-9206-73f109e026ac None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:31.176875 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbab2ec-e93d-4a04-9e7d-cd7ecdfcf85a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:31.214415 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef28cf-d965-4208-a780-9a274027c9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:31.215048 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef28cf-d965-4208-a780-9a274027c9fb None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:31.215372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef28cf-d965-4208-a780-9a274027c9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:31.215629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef28cf-d965-4208-a780-9a274027c9fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:31.221641 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a892c69-ae31-4111-9206-73f109e026ac None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:31.228379 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef28cf-d965-4208-a780-9a274027c9fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:31.230161 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef28cf-d965-4208-a780-9a274027c9fb None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-bbef28cf-d965-4208-a780-9a274027c9fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:31.245601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:31.246722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:31.248138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:31.248138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:31.248138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:31.251429 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 282/555] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:31.252340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef28cf-d965-4208-a780-9a274027c9fb None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:31.268052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644c302-ce44-4dc5-a54b-14ddd67785a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:31.268986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644c302-ce44-4dc5-a54b-14ddd67785a8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:31.269576 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644c302-ce44-4dc5-a54b-14ddd67785a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:31.270364 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644c302-ce44-4dc5-a54b-14ddd67785a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:31.270862 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:31.285351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644c302-ce44-4dc5-a54b-14ddd67785a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:31.287352 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644c302-ce44-4dc5-a54b-14ddd67785a8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-b644c302-ce44-4dc5-a54b-14ddd67785a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:31.306280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:31.307452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:31.308526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:31.308526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:31.308526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:31.314295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 274/556] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:31.315476 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644c302-ce44-4dc5-a54b-14ddd67785a8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:31.348206 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:31.375734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef28cf-d965-4208-a780-9a274027c9fb None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:31.426108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d14f57-3050-418d-bf77-e74095cfd3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:31.427241 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d14f57-3050-418d-bf77-e74095cfd3ba None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:31.427533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d14f57-3050-418d-bf77-e74095cfd3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:31.427812 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d14f57-3050-418d-bf77-e74095cfd3ba None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:31.428910 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d14f57-3050-418d-bf77-e74095cfd3ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:31.429272 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d14f57-3050-418d-bf77-e74095cfd3ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:31.431417 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d14f57-3050-418d-bf77-e74095cfd3ba None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:31.434003 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644c302-ce44-4dc5-a54b-14ddd67785a8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:31.459572 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 283/557] 10.222.0.18 () {56 vars in 1149 bytes} [Fri Oct 14 02:12:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:31.459572 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d14f57-3050-418d-bf77-e74095cfd3ba None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:31.487352 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620a1e9-12ce-467e-bb32-3d92ff9d4ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:31.488639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620a1e9-12ce-467e-bb32-3d92ff9d4ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:31.489312 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620a1e9-12ce-467e-bb32-3d92ff9d4ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:31.489728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620a1e9-12ce-467e-bb32-3d92ff9d4ce1 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:31.490833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620a1e9-12ce-467e-bb32-3d92ff9d4ce1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:31.491688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620a1e9-12ce-467e-bb32-3d92ff9d4ce1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:31.493835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620a1e9-12ce-467e-bb32-3d92ff9d4ce1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:31.509148 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 275/558] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:31.510056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620a1e9-12ce-467e-bb32-3d92ff9d4ce1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:31.542535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:31.578327 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d14f57-3050-418d-bf77-e74095cfd3ba None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:31.603368 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:31.617184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620a1e9-12ce-467e-bb32-3d92ff9d4ce1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:31.621405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320fd30-e4fa-4830-9a07-129290f24b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:31.622508 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320fd30-e4fa-4830-9a07-129290f24b9d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:31.623032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320fd30-e4fa-4830-9a07-129290f24b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:31.623482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320fd30-e4fa-4830-9a07-129290f24b9d None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:31.624320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320fd30-e4fa-4830-9a07-129290f24b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:31.625615 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320fd30-e4fa-4830-9a07-129290f24b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:31.627612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320fd30-e4fa-4830-9a07-129290f24b9d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:31.642839 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 284/559] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:31.644876 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320fd30-e4fa-4830-9a07-129290f24b9d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:31.654186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43519075-534a-4d25-99ac-59ddefd04bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:31.654850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43519075-534a-4d25-99ac-59ddefd04bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:31.655211 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43519075-534a-4d25-99ac-59ddefd04bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:31.655495 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43519075-534a-4d25-99ac-59ddefd04bf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:31.668204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43519075-534a-4d25-99ac-59ddefd04bf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:31.669862 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43519075-534a-4d25-99ac-59ddefd04bf2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-43519075-534a-4d25-99ac-59ddefd04bf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:31.686388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:31.688371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:31.689290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:31.689290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:31.689290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:31.689290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 276/560] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:31.689290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43519075-534a-4d25-99ac-59ddefd04bf2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:31.786286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320fd30-e4fa-4830-9a07-129290f24b9d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:31.833381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:31.842351 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20b5cf-2c68-4553-9fbc-9b702cc76694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:31.843885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20b5cf-2c68-4553-9fbc-9b702cc76694 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:31.844177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20b5cf-2c68-4553-9fbc-9b702cc76694 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:31.844461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20b5cf-2c68-4553-9fbc-9b702cc76694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:31.846153 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43519075-534a-4d25-99ac-59ddefd04bf2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:31.852860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20b5cf-2c68-4553-9fbc-9b702cc76694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:31.854306 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20b5cf-2c68-4553-9fbc-9b702cc76694 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-2c20b5cf-2c68-4553-9fbc-9b702cc76694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:31.868746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:31.869799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:31.870681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:31.870681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:31.870681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:31.870681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:31.870681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:31.870681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:31.870681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:31.870681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:31.870681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 285/561] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:31.871205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20b5cf-2c68-4553-9fbc-9b702cc76694 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:31.892257 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fc66e-47b6-482b-b018-152ebb0a63c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:31.893226 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fc66e-47b6-482b-b018-152ebb0a63c2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:31.893653 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fc66e-47b6-482b-b018-152ebb0a63c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:31.894282 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fc66e-47b6-482b-b018-152ebb0a63c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:31.904952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fc66e-47b6-482b-b018-152ebb0a63c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:31.907000 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fc66e-47b6-482b-b018-152ebb0a63c2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:31.909869 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-642fc66e-47b6-482b-b018-152ebb0a63c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:31.921055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:31.922600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:31.923566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:31.923566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:31.923566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:31.923566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:31.923566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:31.923566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:31.923566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:31.923566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:31.923566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:12:31.923566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:31.923566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 277/562] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Oct 14 02:12:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:31.924160 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fc66e-47b6-482b-b018-152ebb0a63c2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:31.987909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20b5cf-2c68-4553-9fbc-9b702cc76694 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:32.029236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1edee3-7d4a-4ee0-ad5c-57e000a55b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:32.030148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1edee3-7d4a-4ee0-ad5c-57e000a55b4a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:32.030935 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1edee3-7d4a-4ee0-ad5c-57e000a55b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:32.031338 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1edee3-7d4a-4ee0-ad5c-57e000a55b4a None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:32.032279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1edee3-7d4a-4ee0-ad5c-57e000a55b4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:32.032588 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1edee3-7d4a-4ee0-ad5c-57e000a55b4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:32.034658 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1edee3-7d4a-4ee0-ad5c-57e000a55b4a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:32.037149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fc66e-47b6-482b-b018-152ebb0a63c2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:32.051876 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 286/563] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:32.052737 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1edee3-7d4a-4ee0-ad5c-57e000a55b4a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:32.082805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d0952-7d90-4a59-a041-4472334eea5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:32.083886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d0952-7d90-4a59-a041-4472334eea5e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:32.084468 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d0952-7d90-4a59-a041-4472334eea5e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:32.084749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d0952-7d90-4a59-a041-4472334eea5e None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:32.085598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d0952-7d90-4a59-a041-4472334eea5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:32.085908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d0952-7d90-4a59-a041-4472334eea5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:32.087507 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d0952-7d90-4a59-a041-4472334eea5e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:32.107539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 278/564] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:32.109564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d0952-7d90-4a59-a041-4472334eea5e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:32.112703 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:32.167730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1edee3-7d4a-4ee0-ad5c-57e000a55b4a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:32.184394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:32.216863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a28dba-11c5-40e4-999b-e156b0b0dae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:32.217472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a28dba-11c5-40e4-999b-e156b0b0dae0 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:32.217752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a28dba-11c5-40e4-999b-e156b0b0dae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:32.218025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a28dba-11c5-40e4-999b-e156b0b0dae0 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:32.219481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a28dba-11c5-40e4-999b-e156b0b0dae0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:32.219800 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a28dba-11c5-40e4-999b-e156b0b0dae0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:32.221244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a28dba-11c5-40e4-999b-e156b0b0dae0 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:32.242536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d0952-7d90-4a59-a041-4472334eea5e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:32.243420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 287/565] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Oct 14 02:12:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:32.244284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a28dba-11c5-40e4-999b-e156b0b0dae0 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:32.287147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904841b-9b62-4974-9bf6-7ee685b0b4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:32.288522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904841b-9b62-4974-9bf6-7ee685b0b4ad None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:32.288956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904841b-9b62-4974-9bf6-7ee685b0b4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:32.289399 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904841b-9b62-4974-9bf6-7ee685b0b4ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:32.299229 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904841b-9b62-4974-9bf6-7ee685b0b4ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:32.300680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904841b-9b62-4974-9bf6-7ee685b0b4ad None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-8904841b-9b62-4974-9bf6-7ee685b0b4ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:32.312861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:32.313944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:32.314951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:32.314951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:32.314951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:32.314951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:32.314951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:32.314951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:32.314951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:32.314951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:32.314951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 279/566] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:32.320530 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904841b-9b62-4974-9bf6-7ee685b0b4ad None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:32.375868 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a28dba-11c5-40e4-999b-e156b0b0dae0 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:32.388561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:32.418269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0884a9-666a-4071-bd75-38fa5c112630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:32.418972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0884a9-666a-4071-bd75-38fa5c112630 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:32.419284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0884a9-666a-4071-bd75-38fa5c112630 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:32.419630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0884a9-666a-4071-bd75-38fa5c112630 None admin] PATH_INFO: `/v3/projects/f28d337589024cc88184355bd0ee8572/users/cf0efd6d12ad458388e0c3574bbedc6a/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:32.421827 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0884a9-666a-4071-bd75-38fa5c112630 None admin] RBAC: Authorizing `identity:create_grant(project_id=f28d337589024cc88184355bd0ee8572, user_id=cf0efd6d12ad458388e0c3574bbedc6a, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:32.423506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0884a9-666a-4071-bd75-38fa5c112630 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:32.447101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 288/567] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:12:32 2022] PUT /identity/v3/projects/f28d337589024cc88184355bd0ee8572/users/cf0efd6d12ad458388e0c3574bbedc6a/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:12:32.447101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0884a9-666a-4071-bd75-38fa5c112630 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:32.447772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904841b-9b62-4974-9bf6-7ee685b0b4ad None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:32.498108 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:32.529258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8812953-054f-49ae-84e8-21765fd48d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:32.530022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8812953-054f-49ae-84e8-21765fd48d54 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:32.530308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8812953-054f-49ae-84e8-21765fd48d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:32.530580 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8812953-054f-49ae-84e8-21765fd48d54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:32.649623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0884a9-666a-4071-bd75-38fa5c112630 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:32.690964 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb81229-9799-4ec8-b741-8f42f0908caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:32.691534 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb81229-9799-4ec8-b741-8f42f0908caa None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:32.691806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb81229-9799-4ec8-b741-8f42f0908caa None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:32.692074 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb81229-9799-4ec8-b741-8f42f0908caa None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:32.692774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb81229-9799-4ec8-b741-8f42f0908caa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:32.693075 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb81229-9799-4ec8-b741-8f42f0908caa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:32.694615 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb81229-9799-4ec8-b741-8f42f0908caa None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:32.698203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8812953-054f-49ae-84e8-21765fd48d54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:32.699830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8812953-054f-49ae-84e8-21765fd48d54 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:32.708786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 289/568] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:32.714743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e435910-dfe9-46d7-8675-14b0a7332c74 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:32.715033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e435910-dfe9-46d7-8675-14b0a7332c74 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:32.715335 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e435910-dfe9-46d7-8675-14b0a7332c74 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:32.716753 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 290/569] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:32.718111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e435910-dfe9-46d7-8675-14b0a7332c74 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:32.754909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:32.840347 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 280/570] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:12:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:12:32.841687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233ee98-541b-4e13-9b6d-723b42747142 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:32.841970 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233ee98-541b-4e13-9b6d-723b42747142 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:32.842243 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233ee98-541b-4e13-9b6d-723b42747142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:32.863381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1233ee98-541b-4e13-9b6d-723b42747142 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:32.863992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e435910-dfe9-46d7-8675-14b0a7332c74 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:32.864801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1233ee98-541b-4e13-9b6d-723b42747142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:32.881276 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:32.914397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b81f5-e257-44dd-ae98-e5132f862096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:32.915136 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b81f5-e257-44dd-ae98-e5132f862096 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:32.915424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b81f5-e257-44dd-ae98-e5132f862096 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:32.916486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b81f5-e257-44dd-ae98-e5132f862096 None admin] PATH_INFO: `/v3/projects/f28d337589024cc88184355bd0ee8572/users/de87b424a83f48f09fd6c89c5f5621ed/roles/8adf2afafb0446a6992e9aeb380f645d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:32.939261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b81f5-e257-44dd-ae98-e5132f862096 None admin] RBAC: Authorizing `identity:create_grant(project_id=f28d337589024cc88184355bd0ee8572, user_id=de87b424a83f48f09fd6c89c5f5621ed, role_id=8adf2afafb0446a6992e9aeb380f645d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:32.941226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b81f5-e257-44dd-ae98-e5132f862096 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:32.949735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 281/571] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:32.951060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233ee98-541b-4e13-9b6d-723b42747142 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:32.958073 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 291/572] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:12:32 2022] PUT /identity/v3/projects/f28d337589024cc88184355bd0ee8572/users/de87b424a83f48f09fd6c89c5f5621ed/roles/8adf2afafb0446a6992e9aeb380f645d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:12:33.024909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d3ce2-bc81-4ab0-afb4-2c7afeca18af None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:33.025237 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d3ce2-bc81-4ab0-afb4-2c7afeca18af None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:33.025460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d3ce2-bc81-4ab0-afb4-2c7afeca18af None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:33.027132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 292/573] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:12:33.032948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a62f5b-9998-4f81-b261-b38165cd83fc None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:33.033549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a62f5b-9998-4f81-b261-b38165cd83fc None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:33.034038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a62f5b-9998-4f81-b261-b38165cd83fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:33.076213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b4a62f5b-9998-4f81-b261-b38165cd83fc None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:33.077884 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b4a62f5b-9998-4f81-b261-b38165cd83fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:33.079353 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233ee98-541b-4e13-9b6d-723b42747142 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:33.119894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edc8c3-84e4-4b50-a4f3-a2a7011b3ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:33.120554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edc8c3-84e4-4b50-a4f3-a2a7011b3ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:33.121005 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edc8c3-84e4-4b50-a4f3-a2a7011b3ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:33.121286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edc8c3-84e4-4b50-a4f3-a2a7011b3ba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:33.134727 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edc8c3-84e4-4b50-a4f3-a2a7011b3ba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:33.136239 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edc8c3-84e4-4b50-a4f3-a2a7011b3ba8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-f4edc8c3-84e4-4b50-a4f3-a2a7011b3ba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:33.153129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:33.154203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:33.155087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:33.155087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:33.155087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:33.155087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 282/574] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:33.161001 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edc8c3-84e4-4b50-a4f3-a2a7011b3ba8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:33.165347 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:33.189857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 293/575] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:12:33.273365 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edc8c3-84e4-4b50-a4f3-a2a7011b3ba8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:33.303771 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a62f5b-9998-4f81-b261-b38165cd83fc None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:33.308767 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac341f4-5654-4be7-a3f7-d3d62190ca01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:33.309402 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac341f4-5654-4be7-a3f7-d3d62190ca01 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:33.309680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac341f4-5654-4be7-a3f7-d3d62190ca01 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:33.309960 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac341f4-5654-4be7-a3f7-d3d62190ca01 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:33.310613 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac341f4-5654-4be7-a3f7-d3d62190ca01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:33.310921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac341f4-5654-4be7-a3f7-d3d62190ca01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:33.313118 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac341f4-5654-4be7-a3f7-d3d62190ca01 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:33.319002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:33.328033 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 283/576] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:33.333508 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac341f4-5654-4be7-a3f7-d3d62190ca01 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:33.403433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:33.455893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a62f5b-9998-4f81-b261-b38165cd83fc None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:33.461902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac341f4-5654-4be7-a3f7-d3d62190ca01 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:33.495133 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614e59a-9df4-4ec0-bb94-6beefe7d402b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:33.495943 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614e59a-9df4-4ec0-bb94-6beefe7d402b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:33.497324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614e59a-9df4-4ec0-bb94-6beefe7d402b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:33.500475 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614e59a-9df4-4ec0-bb94-6beefe7d402b None admin] PATH_INFO: `/v3/services/network` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:33.501181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1614e59a-9df4-4ec0-bb94-6beefe7d402b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:33.502660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1614e59a-9df4-4ec0-bb94-6beefe7d402b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-1614e59a-9df4-4ec0-bb94-6beefe7d402b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:33.515962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:33.517090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:33.518053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:33.518053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:33.518053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:33.518053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:33.518053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:33.518053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:33.518053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:33.518053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:33.518053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:33.518053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:33.518053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:33.518053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:33.518053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:12:33.518053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:33.522080 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 294/577] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:33 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:33.528548 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614e59a-9df4-4ec0-bb94-6beefe7d402b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:33.532387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd833c5-81a8-438f-b03b-b617b2e1654a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:33.534893 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd833c5-81a8-438f-b03b-b617b2e1654a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:33.535629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd833c5-81a8-438f-b03b-b617b2e1654a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:33.537678 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd833c5-81a8-438f-b03b-b617b2e1654a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:33.556446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd833c5-81a8-438f-b03b-b617b2e1654a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:33.558411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd833c5-81a8-438f-b03b-b617b2e1654a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-3cd833c5-81a8-438f-b03b-b617b2e1654a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:33.576975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:33.578704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:33.579830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:33.579830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:33.579830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:33.579830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:33.579830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:33.579830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:33.579830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:33.579830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:33.581746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 284/578] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:33.582792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd833c5-81a8-438f-b03b-b617b2e1654a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:33.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:33.648257 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614e59a-9df4-4ec0-bb94-6beefe7d402b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:33.684103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c855e2-281e-46cc-a2a8-0765f9234661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:33.684909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c855e2-281e-46cc-a2a8-0765f9234661 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:33.685286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c855e2-281e-46cc-a2a8-0765f9234661 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:33.685661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c855e2-281e-46cc-a2a8-0765f9234661 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:33.686498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c855e2-281e-46cc-a2a8-0765f9234661 None admin] RBAC: Adding query filter params (name=network) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:33.686842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c855e2-281e-46cc-a2a8-0765f9234661 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:33.688347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c855e2-281e-46cc-a2a8-0765f9234661 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:33.697657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 295/579] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Oct 14 02:12:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:33.697657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c855e2-281e-46cc-a2a8-0765f9234661 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:33.698610 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd833c5-81a8-438f-b03b-b617b2e1654a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:33.728128 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:33.736877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a3e14-95c5-4cb0-9b37-40fb575c1e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:33.737670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a3e14-95c5-4cb0-9b37-40fb575c1e6c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:33.738085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a3e14-95c5-4cb0-9b37-40fb575c1e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:33.740696 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a3e14-95c5-4cb0-9b37-40fb575c1e6c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:33.741448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a3e14-95c5-4cb0-9b37-40fb575c1e6c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:33.743105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a3e14-95c5-4cb0-9b37-40fb575c1e6c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-e05a3e14-95c5-4cb0-9b37-40fb575c1e6c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:33.753297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:33.754497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:33.755548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:33.755548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:33.755548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:33.755548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:33.755548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:33.755548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:33.755548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:33.755548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:33.755548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:33.755548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:33.755548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:33.755548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:33.755548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:12:33.755548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:33.762670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 285/580] 10.222.0.18 () {56 vars in 1129 bytes} [Fri Oct 14 02:12:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:33.763832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a3e14-95c5-4cb0-9b37-40fb575c1e6c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:33.808667 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c855e2-281e-46cc-a2a8-0765f9234661 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:33.844026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c1e57-c34a-49e2-a8dc-e899bd2a847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:33.844794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c1e57-c34a-49e2-a8dc-e899bd2a847c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:33.845182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c1e57-c34a-49e2-a8dc-e899bd2a847c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:33.845558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c1e57-c34a-49e2-a8dc-e899bd2a847c None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:33.846627 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c1e57-c34a-49e2-a8dc-e899bd2a847c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:33.846752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c1e57-c34a-49e2-a8dc-e899bd2a847c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:33.848325 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c1e57-c34a-49e2-a8dc-e899bd2a847c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:33.850948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:33.861875 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 296/581] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:33.862858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c1e57-c34a-49e2-a8dc-e899bd2a847c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:33.876167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a3e14-95c5-4cb0-9b37-40fb575c1e6c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:33.913298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42793e28-3e82-43f3-bfb8-d7425f5c5fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:33.914195 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42793e28-3e82-43f3-bfb8-d7425f5c5fbc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:33.914612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42793e28-3e82-43f3-bfb8-d7425f5c5fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:33.915023 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42793e28-3e82-43f3-bfb8-d7425f5c5fbc None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:33.915938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42793e28-3e82-43f3-bfb8-d7425f5c5fbc None admin] RBAC: Adding query filter params (type=network) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:33.916380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42793e28-3e82-43f3-bfb8-d7425f5c5fbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:33.917924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42793e28-3e82-43f3-bfb8-d7425f5c5fbc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:33.929503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 286/582] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Oct 14 02:12:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:33.930666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42793e28-3e82-43f3-bfb8-d7425f5c5fbc None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:33.970136 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c1e57-c34a-49e2-a8dc-e899bd2a847c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:33.986442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:34.008628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c498d-3d09-47f4-a4a5-cd68d2cde0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:34.010111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c498d-3d09-47f4-a4a5-cd68d2cde0cf None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:34.010403 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c498d-3d09-47f4-a4a5-cd68d2cde0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:34.010677 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c498d-3d09-47f4-a4a5-cd68d2cde0cf None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:34.011353 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c498d-3d09-47f4-a4a5-cd68d2cde0cf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:34.011660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c498d-3d09-47f4-a4a5-cd68d2cde0cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:34.013033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c498d-3d09-47f4-a4a5-cd68d2cde0cf None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:34.023817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 297/583] 10.222.0.18 () {56 vars in 1124 bytes} [Fri Oct 14 02:12:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:34.023817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c498d-3d09-47f4-a4a5-cd68d2cde0cf None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:34.048749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42793e28-3e82-43f3-bfb8-d7425f5c5fbc None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:34.086259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa72956-06b9-48a2-82bd-f5c9dd044f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:34.087167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa72956-06b9-48a2-82bd-f5c9dd044f6f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:34.087632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa72956-06b9-48a2-82bd-f5c9dd044f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:34.088065 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa72956-06b9-48a2-82bd-f5c9dd044f6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:34.088921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa72956-06b9-48a2-82bd-f5c9dd044f6f None admin] RBAC: Adding query filter params (role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b, scope.project.id=f28d337589024cc88184355bd0ee8572, user.id=de13be55b4034477baf4c029c1a79757) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:34.089367 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa72956-06b9-48a2-82bd-f5c9dd044f6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:34.091221 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa72956-06b9-48a2-82bd-f5c9dd044f6f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:34.094097 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:34.100171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 287/584] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:12:33 2022] GET /identity/v3/role_assignments?user.id=de13be55b4034477baf4c029c1a79757&scope.project.id=f28d337589024cc88184355bd0ee8572&role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:34.101298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa72956-06b9-48a2-82bd-f5c9dd044f6f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:34.153570 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c498d-3d09-47f4-a4a5-cd68d2cde0cf None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:34.205946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7c624-25b1-469c-98b7-4d15b96dd920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:34.206903 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7c624-25b1-469c-98b7-4d15b96dd920 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:34.207221 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7c624-25b1-469c-98b7-4d15b96dd920 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:34.207496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7c624-25b1-469c-98b7-4d15b96dd920 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:34.208215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7c624-25b1-469c-98b7-4d15b96dd920 None admin] RBAC: Adding query filter params (interface=public, service_id=2b80e727f66c4e7cb503426887e2f4d9, region_id=RegionOne) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:34.208524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7c624-25b1-469c-98b7-4d15b96dd920 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:34.210629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7c624-25b1-469c-98b7-4d15b96dd920 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:34.219005 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa72956-06b9-48a2-82bd-f5c9dd044f6f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:34.226366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 298/585] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Oct 14 02:12:34 2022] GET /identity/v3/endpoints?service_id=2b80e727f66c4e7cb503426887e2f4d9&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:34.232628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7c624-25b1-469c-98b7-4d15b96dd920 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:34.250342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:34.286608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725c1ce-1e4c-4e45-b82d-22333978e442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:34.287253 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725c1ce-1e4c-4e45-b82d-22333978e442 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:34.287532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725c1ce-1e4c-4e45-b82d-22333978e442 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:34.287801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725c1ce-1e4c-4e45-b82d-22333978e442 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:34.288436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725c1ce-1e4c-4e45-b82d-22333978e442 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:34.288736 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725c1ce-1e4c-4e45-b82d-22333978e442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:34.290104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725c1ce-1e4c-4e45-b82d-22333978e442 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:34.301082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 288/586] 10.222.0.18 () {56 vars in 1124 bytes} [Fri Oct 14 02:12:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:34.302468 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725c1ce-1e4c-4e45-b82d-22333978e442 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:34.434912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725c1ce-1e4c-4e45-b82d-22333978e442 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:34.437803 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7c624-25b1-469c-98b7-4d15b96dd920 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:34.463820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:34.472750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11616c10-f915-4c63-9281-715748eb6e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:34.473209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11616c10-f915-4c63-9281-715748eb6e24 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:34.473580 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11616c10-f915-4c63-9281-715748eb6e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:34.473978 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11616c10-f915-4c63-9281-715748eb6e24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:34.474685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11616c10-f915-4c63-9281-715748eb6e24 None admin] RBAC: Adding query filter params (interface=public, service_id=b7813a166f89496b881aa0d7a128e870, region_id=RegionOne) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:34.475839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11616c10-f915-4c63-9281-715748eb6e24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:34.476516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11616c10-f915-4c63-9281-715748eb6e24 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:34.480831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae5012-65a7-418d-8579-5241d0d88870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:34.481717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae5012-65a7-418d-8579-5241d0d88870 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:34.482192 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae5012-65a7-418d-8579-5241d0d88870 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:34.482655 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae5012-65a7-418d-8579-5241d0d88870 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:34.483472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae5012-65a7-418d-8579-5241d0d88870 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:34.485079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae5012-65a7-418d-8579-5241d0d88870 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:34.487372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 289/587] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Oct 14 02:12:34 2022] GET /identity/v3/endpoints?service_id=b7813a166f89496b881aa0d7a128e870&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:34.490635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11616c10-f915-4c63-9281-715748eb6e24 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:34.491942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 299/588] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Oct 14 02:12:34 2022] GET /identity/v3/services => generated 772 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:34.542139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:34.607949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11616c10-f915-4c63-9281-715748eb6e24 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:34.646050 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af808503-4c01-40d9-b033-cb0143521f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:34.646726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af808503-4c01-40d9-b033-cb0143521f2e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:34.647008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af808503-4c01-40d9-b033-cb0143521f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:34.647310 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af808503-4c01-40d9-b033-cb0143521f2e None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:34.647929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af808503-4c01-40d9-b033-cb0143521f2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:34.649354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af808503-4c01-40d9-b033-cb0143521f2e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:34.660526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 290/589] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Oct 14 02:12:34 2022] GET /identity/v3/services => generated 772 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:34.953886 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f911d-9c4b-4aa3-a76b-5cb676955acf None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:34.954061 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f911d-9c4b-4aa3-a76b-5cb676955acf None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:34.954321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f911d-9c4b-4aa3-a76b-5cb676955acf None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:34.956010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 300/590] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:34.961281 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f2a09-b597-46f9-aa62-d23a1be9b2f2 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:34.961753 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f2a09-b597-46f9-aa62-d23a1be9b2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:34.961830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f2a09-b597-46f9-aa62-d23a1be9b2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:34.984698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-603f2a09-b597-46f9-aa62-d23a1be9b2f2 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:34.985917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-603f2a09-b597-46f9-aa62-d23a1be9b2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:35.079434 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 291/591] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:12:35.582975 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f911d-9c4b-4aa3-a76b-5cb676955acf None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:35.748166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f911d-9c4b-4aa3-a76b-5cb676955acf None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:35.793737 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b866b19-c286-4434-8f95-b3b7a4a5305a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:35.794380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b866b19-c286-4434-8f95-b3b7a4a5305a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:35.794655 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b866b19-c286-4434-8f95-b3b7a4a5305a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:35.794924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b866b19-c286-4434-8f95-b3b7a4a5305a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:35.812622 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b866b19-c286-4434-8f95-b3b7a4a5305a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:35.819187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f7d4d-f0c4-4052-b7b6-f878c4c89407 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:35.819480 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f7d4d-f0c4-4052-b7b6-f878c4c89407 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:35.819758 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f7d4d-f0c4-4052-b7b6-f878c4c89407 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-4b866b19-c286-4434-8f95-b3b7a4a5305a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:35.820625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:35.821580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:35.822456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:35.822456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:35.822456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:35.822456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:35.822456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:35.822456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:35.822456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:35.822456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:35.822456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:35.822456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:35.822456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:12:35.822456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:35.823153 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 292/592] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:35.823231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 301/593] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:12:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:35.826270 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e9ad-55ed-40eb-b977-5bc7a1a7ff3f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:35.826556 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e9ad-55ed-40eb-b977-5bc7a1a7ff3f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:35.826837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e9ad-55ed-40eb-b977-5bc7a1a7ff3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:35.828473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b866b19-c286-4434-8f95-b3b7a4a5305a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:35.846410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f3f0e9ad-55ed-40eb-b977-5bc7a1a7ff3f None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:35.847565 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f3f0e9ad-55ed-40eb-b977-5bc7a1a7ff3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:35.907702 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:35.921916 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:35.985781 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 293/594] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:36.017464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b866b19-c286-4434-8f95-b3b7a4a5305a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:36.058158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7405a4e-a327-42aa-95a0-de7945aa9470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:36.058768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7405a4e-a327-42aa-95a0-de7945aa9470 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:36.059045 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7405a4e-a327-42aa-95a0-de7945aa9470 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:36.059339 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7405a4e-a327-42aa-95a0-de7945aa9470 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:36.060010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7405a4e-a327-42aa-95a0-de7945aa9470 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:36.060311 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7405a4e-a327-42aa-95a0-de7945aa9470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:36.062219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7405a4e-a327-42aa-95a0-de7945aa9470 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:36.070952 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 302/595] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:12:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:36.076628 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0e9ad-55ed-40eb-b977-5bc7a1a7ff3f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:36.202996 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0e9ad-55ed-40eb-b977-5bc7a1a7ff3f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:36.249894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458c2bd-14de-40e9-8a85-e0a7e1d621cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:36.250924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458c2bd-14de-40e9-8a85-e0a7e1d621cb None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:36.251434 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458c2bd-14de-40e9-8a85-e0a7e1d621cb None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:36.251916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458c2bd-14de-40e9-8a85-e0a7e1d621cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:36.268789 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458c2bd-14de-40e9-8a85-e0a7e1d621cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:36.270637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458c2bd-14de-40e9-8a85-e0a7e1d621cb None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-7458c2bd-14de-40e9-8a85-e0a7e1d621cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:36.289804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:36.290764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:36.291678 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:36.291678 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:36.291678 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:36.293716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 294/596] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:36.300141 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7405a4e-a327-42aa-95a0-de7945aa9470 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:36.378342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458c2bd-14de-40e9-8a85-e0a7e1d621cb None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:36.421606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7405a4e-a327-42aa-95a0-de7945aa9470 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:36.423054 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:36.433462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:36.469773 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50926b1-61ca-4873-a223-1fced1aaa635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:36.470639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50926b1-61ca-4873-a223-1fced1aaa635 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:36.471100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50926b1-61ca-4873-a223-1fced1aaa635 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:36.471535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50926b1-61ca-4873-a223-1fced1aaa635 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:36.472341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50926b1-61ca-4873-a223-1fced1aaa635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:36.472796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50926b1-61ca-4873-a223-1fced1aaa635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:36.474314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50926b1-61ca-4873-a223-1fced1aaa635 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:36.496747 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 303/597] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:36.502786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50926b1-61ca-4873-a223-1fced1aaa635 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:36.552673 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458c2bd-14de-40e9-8a85-e0a7e1d621cb None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:36.621571 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c65e57-0fc2-4b65-8237-a0b77d93a80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:36.622465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c65e57-0fc2-4b65-8237-a0b77d93a80d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:36.622745 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c65e57-0fc2-4b65-8237-a0b77d93a80d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:36.623019 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c65e57-0fc2-4b65-8237-a0b77d93a80d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:36.653840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c65e57-0fc2-4b65-8237-a0b77d93a80d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-40c65e57-0fc2-4b65-8237-a0b77d93a80d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:36.660129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:36.661039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:36.661919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:36.661919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:36.661919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:36.661919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:36.661919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:36.661919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:36.661919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:36.661919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:36.661919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:36.661919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:36.661919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:12:36.661919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:36.662578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50926b1-61ca-4873-a223-1fced1aaa635 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:36.662978 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 295/598] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:12:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:36.668446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c65e57-0fc2-4b65-8237-a0b77d93a80d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:36.703744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d4c4d-3e2c-454d-88a2-4e1b24c94d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:36.705072 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d4c4d-3e2c-454d-88a2-4e1b24c94d3b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:36.705681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d4c4d-3e2c-454d-88a2-4e1b24c94d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:36.706158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d4c4d-3e2c-454d-88a2-4e1b24c94d3b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:36.713899 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d4c4d-3e2c-454d-88a2-4e1b24c94d3b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:36.715603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d4c4d-3e2c-454d-88a2-4e1b24c94d3b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-867d4c4d-3e2c-454d-88a2-4e1b24c94d3b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:36.730288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:36.731217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:36.732142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:36.732142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:36.732142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:36.732142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:36.732142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:36.732142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:36.732142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:36.732142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:36.732142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:12:36.732142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:36.735348 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 304/599] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:12:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:36.750275 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d4c4d-3e2c-454d-88a2-4e1b24c94d3b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:36.792194 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:36.835830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:36.863378 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c65e57-0fc2-4b65-8237-a0b77d93a80d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:36.898263 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf11988-6cc3-4ecc-9ed9-79041a039562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:36.898878 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf11988-6cc3-4ecc-9ed9-79041a039562 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:36.899231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf11988-6cc3-4ecc-9ed9-79041a039562 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:36.899510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf11988-6cc3-4ecc-9ed9-79041a039562 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:36.900178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf11988-6cc3-4ecc-9ed9-79041a039562 None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:36.900481 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf11988-6cc3-4ecc-9ed9-79041a039562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:36.901886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf11988-6cc3-4ecc-9ed9-79041a039562 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:36.916109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 296/600] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Oct 14 02:12:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:36.917498 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b149fd9-3f3d-49d3-8b71-f51c0fc12704 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:36.917785 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b149fd9-3f3d-49d3-8b71-f51c0fc12704 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:36.918059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b149fd9-3f3d-49d3-8b71-f51c0fc12704 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:36.919836 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 297/601] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:36.920650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b149fd9-3f3d-49d3-8b71-f51c0fc12704 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:36.935502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d4c4d-3e2c-454d-88a2-4e1b24c94d3b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:36.974253 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f844c84-42a8-4e05-9c4b-9a9dbae780ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:36.975277 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f844c84-42a8-4e05-9c4b-9a9dbae780ac None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:36.975935 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f844c84-42a8-4e05-9c4b-9a9dbae780ac None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:36.976407 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f844c84-42a8-4e05-9c4b-9a9dbae780ac None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:36.977345 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f844c84-42a8-4e05-9c4b-9a9dbae780ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:36.977842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f844c84-42a8-4e05-9c4b-9a9dbae780ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:36.979691 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f844c84-42a8-4e05-9c4b-9a9dbae780ac None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:36.999888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 305/602] 10.222.0.18 () {56 vars in 1135 bytes} [Fri Oct 14 02:12:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:37.001444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3057e8-9684-4472-841a-a24dd164f6c2 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:37.002044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3057e8-9684-4472-841a-a24dd164f6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:37.002531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3057e8-9684-4472-841a-a24dd164f6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:37.023588 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-df3057e8-9684-4472-841a-a24dd164f6c2 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:37.025254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-df3057e8-9684-4472-841a-a24dd164f6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:37.029807 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b149fd9-3f3d-49d3-8b71-f51c0fc12704 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:37.065983 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c231c6a-8ae4-4fce-b904-7d4567f904e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:37.066623 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c231c6a-8ae4-4fce-b904-7d4567f904e8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:37.066899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c231c6a-8ae4-4fce-b904-7d4567f904e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:37.067199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c231c6a-8ae4-4fce-b904-7d4567f904e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:37.077396 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:37.079792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c231c6a-8ae4-4fce-b904-7d4567f904e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:37.081325 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c231c6a-8ae4-4fce-b904-7d4567f904e8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-9c231c6a-8ae4-4fce-b904-7d4567f904e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:37.096483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:37.097436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:37.098302 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:37.098302 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:37.098302 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:37.102258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 298/603] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:37.102943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c231c6a-8ae4-4fce-b904-7d4567f904e8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:37.107220 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 306/604] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:37.108366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3057e8-9684-4472-841a-a24dd164f6c2 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:37.128495 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:37.206275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c231c6a-8ae4-4fce-b904-7d4567f904e8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:37.230297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3057e8-9684-4472-841a-a24dd164f6c2 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:37.249557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab45c1-5a22-40a7-a36c-37fdf3070b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:37.250184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab45c1-5a22-40a7-a36c-37fdf3070b7e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:37.250462 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab45c1-5a22-40a7-a36c-37fdf3070b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:37.250777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab45c1-5a22-40a7-a36c-37fdf3070b7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:37.275610 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c6a0e-b2d5-4926-9f19-43ec2c9c7294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:37.276571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c6a0e-b2d5-4926-9f19-43ec2c9c7294 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:37.277062 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c6a0e-b2d5-4926-9f19-43ec2c9c7294 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:37.277530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c6a0e-b2d5-4926-9f19-43ec2c9c7294 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:37.278384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c6a0e-b2d5-4926-9f19-43ec2c9c7294 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:37.278898 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c6a0e-b2d5-4926-9f19-43ec2c9c7294 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:37.280559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c6a0e-b2d5-4926-9f19-43ec2c9c7294 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:37.293205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 307/605] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:37.299253 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c6a0e-b2d5-4926-9f19-43ec2c9c7294 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:37.390284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:37.397526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:37.418138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab45c1-5a22-40a7-a36c-37fdf3070b7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:37.419340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c6a0e-b2d5-4926-9f19-43ec2c9c7294 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:37.419777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab45c1-5a22-40a7-a36c-37fdf3070b7e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:37.466074 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b5438-366b-4e99-95d2-265a1ff69a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:37.466708 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b5438-366b-4e99-95d2-265a1ff69a14 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:37.466985 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b5438-366b-4e99-95d2-265a1ff69a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:37.467287 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b5438-366b-4e99-95d2-265a1ff69a14 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:37.478432 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b5438-366b-4e99-95d2-265a1ff69a14 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:37.479903 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b5438-366b-4e99-95d2-265a1ff69a14 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-560b5438-366b-4e99-95d2-265a1ff69a14 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:37.492132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:37.493050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:37.493963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:37.493963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:37.493963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:37.493963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:37.493963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:37.493963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:37.493963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:37.493963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:37.493963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:12:37.493963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:37.493963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 308/606] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Oct 14 02:12:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:37.493963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b5438-366b-4e99-95d2-265a1ff69a14 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:37.537419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 299/607] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:12:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:12:37.539093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89d7f5-0ef3-4ed1-bec6-471bd9de8a4d None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:37.539384 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89d7f5-0ef3-4ed1-bec6-471bd9de8a4d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:37.539659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89d7f5-0ef3-4ed1-bec6-471bd9de8a4d None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:37.540902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 300/608] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:37.541601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89d7f5-0ef3-4ed1-bec6-471bd9de8a4d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:37.600927 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b5438-366b-4e99-95d2-265a1ff69a14 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:37.630289 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:37.634422 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc91210-b1dd-4337-b1e1-b5340940563b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:37.635050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc91210-b1dd-4337-b1e1-b5340940563b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:37.635353 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc91210-b1dd-4337-b1e1-b5340940563b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:37.635622 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc91210-b1dd-4337-b1e1-b5340940563b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:37.636195 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc91210-b1dd-4337-b1e1-b5340940563b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:37.637544 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc91210-b1dd-4337-b1e1-b5340940563b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:37.640773 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89d7f5-0ef3-4ed1-bec6-471bd9de8a4d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-1dc91210-b1dd-4337-b1e1-b5340940563b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:37.646575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:37.647734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:37.648612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 309/609] 10.222.0.18 () {56 vars in 1132 bytes} [Fri Oct 14 02:12:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:37.649423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1c1bd-4997-436b-aeb2-b66288ccff5b None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:37.649423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1c1bd-4997-436b-aeb2-b66288ccff5b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:37.649423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1c1bd-4997-436b-aeb2-b66288ccff5b None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:37.650495 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 310/610] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:37.651552 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1c1bd-4997-436b-aeb2-b66288ccff5b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:37.656318 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:37.677745 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732b119-acba-4a09-b4b0-959a84d36d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:37.678351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732b119-acba-4a09-b4b0-959a84d36d60 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:37.678625 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732b119-acba-4a09-b4b0-959a84d36d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:37.678896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732b119-acba-4a09-b4b0-959a84d36d60 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:37.679648 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732b119-acba-4a09-b4b0-959a84d36d60 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:37.679946 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732b119-acba-4a09-b4b0-959a84d36d60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:37.681370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732b119-acba-4a09-b4b0-959a84d36d60 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:37.698460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 301/611] 10.222.0.18 () {56 vars in 1149 bytes} [Fri Oct 14 02:12:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:37.699660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27089795-757a-4b26-b2c1-a894c6fbb104 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:37.699938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27089795-757a-4b26-b2c1-a894c6fbb104 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:37.700206 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27089795-757a-4b26-b2c1-a894c6fbb104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:37.717586 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-27089795-757a-4b26-b2c1-a894c6fbb104 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:37.719456 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-27089795-757a-4b26-b2c1-a894c6fbb104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:37.760298 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1c1bd-4997-436b-aeb2-b66288ccff5b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:37.785277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 302/612] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:37.786116 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27089795-757a-4b26-b2c1-a894c6fbb104 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:37.809335 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae24ce-eccd-4543-aba6-d9a6de3afc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:37.809963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae24ce-eccd-4543-aba6-d9a6de3afc99 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:37.810246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae24ce-eccd-4543-aba6-d9a6de3afc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:37.810521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae24ce-eccd-4543-aba6-d9a6de3afc99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:37.821352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae24ce-eccd-4543-aba6-d9a6de3afc99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:37.822796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae24ce-eccd-4543-aba6-d9a6de3afc99 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-77ae24ce-eccd-4543-aba6-d9a6de3afc99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:37.838205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:37.839214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:37.840093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:37.840093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:37.840093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:37.840093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 311/613] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:37.840812 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefec23-34d9-4d97-acf7-128f18b166f0 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:37.840812 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefec23-34d9-4d97-acf7-128f18b166f0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:37.840949 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefec23-34d9-4d97-acf7-128f18b166f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:37.859578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-afefec23-34d9-4d97-acf7-128f18b166f0 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:37.860848 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-afefec23-34d9-4d97-acf7-128f18b166f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:37.873582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:37.895102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27089795-757a-4b26-b2c1-a894c6fbb104 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:37.913111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:37.932303 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a33687-3364-4bb0-9de6-ce1bf88bb0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:37.932938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a33687-3364-4bb0-9de6-ce1bf88bb0fa None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:37.933220 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a33687-3364-4bb0-9de6-ce1bf88bb0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:37.933496 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a33687-3364-4bb0-9de6-ce1bf88bb0fa None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:37.934127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a33687-3364-4bb0-9de6-ce1bf88bb0fa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:37.934430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a33687-3364-4bb0-9de6-ce1bf88bb0fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:37.935859 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a33687-3364-4bb0-9de6-ce1bf88bb0fa None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:37.936062 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 312/614] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:37.937267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefec23-34d9-4d97-acf7-128f18b166f0 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:37.951048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 303/615] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Oct 14 02:12:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:37.952702 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a33687-3364-4bb0-9de6-ce1bf88bb0fa None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:38.058702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefec23-34d9-4d97-acf7-128f18b166f0 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:38.075686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a33687-3364-4bb0-9de6-ce1bf88bb0fa None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:38.096604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8422d7a-b592-4622-b418-61ea1da951d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:38.097236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8422d7a-b592-4622-b418-61ea1da951d5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:38.097512 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8422d7a-b592-4622-b418-61ea1da951d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:38.097782 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8422d7a-b592-4622-b418-61ea1da951d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:38.114982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97513865-97e0-4659-9901-746515496664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:38.116021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97513865-97e0-4659-9901-746515496664 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:38.116527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97513865-97e0-4659-9901-746515496664 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:38.116869 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97513865-97e0-4659-9901-746515496664 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:38.117573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97513865-97e0-4659-9901-746515496664 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:38.117878 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97513865-97e0-4659-9901-746515496664 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:38.119523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97513865-97e0-4659-9901-746515496664 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:38.138694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 304/616] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:38.140019 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97513865-97e0-4659-9901-746515496664 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:38.143577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:38.152135 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:38.169928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8422d7a-b592-4622-b418-61ea1da951d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:38.171726 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8422d7a-b592-4622-b418-61ea1da951d5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:38.248307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97513865-97e0-4659-9901-746515496664 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:38.269878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 313/617] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:12:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:12:38.270713 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8422d7a-b592-4622-b418-61ea1da951d5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:38.282945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a5cff-89aa-4106-8e6c-fc3f42e86fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:38.283598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a5cff-89aa-4106-8e6c-fc3f42e86fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:38.284197 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a5cff-89aa-4106-8e6c-fc3f42e86fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:38.284484 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a5cff-89aa-4106-8e6c-fc3f42e86fa0 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:38.285131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a5cff-89aa-4106-8e6c-fc3f42e86fa0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:38.285436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a5cff-89aa-4106-8e6c-fc3f42e86fa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:38.286801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a5cff-89aa-4106-8e6c-fc3f42e86fa0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:38.291182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 305/618] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Oct 14 02:12:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:38.291961 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a5cff-89aa-4106-8e6c-fc3f42e86fa0 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:38.362184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:38.382053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:38.382767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8422d7a-b592-4622-b418-61ea1da951d5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:38.419676 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11410e2-5efa-4842-ad6c-2055b401c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:38.420341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11410e2-5efa-4842-ad6c-2055b401c7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:38.420623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11410e2-5efa-4842-ad6c-2055b401c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:38.420995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11410e2-5efa-4842-ad6c-2055b401c7c6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:38.422165 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11410e2-5efa-4842-ad6c-2055b401c7c6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:38.423768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11410e2-5efa-4842-ad6c-2055b401c7c6 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:38.424735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a5cff-89aa-4106-8e6c-fc3f42e86fa0 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-c11410e2-5efa-4842-ad6c-2055b401c7c6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:38.433324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:38.434355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:38.435209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 314/619] 10.222.0.18 () {56 vars in 1129 bytes} [Fri Oct 14 02:12:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:38.436763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11410e2-5efa-4842-ad6c-2055b401c7c6 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:38.467562 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32564c44-e940-47b9-b6de-ef8233fb6bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:38.469308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32564c44-e940-47b9-b6de-ef8233fb6bbd None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:38.469767 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32564c44-e940-47b9-b6de-ef8233fb6bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:38.470244 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32564c44-e940-47b9-b6de-ef8233fb6bbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:38.478927 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32564c44-e940-47b9-b6de-ef8233fb6bbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:38.480465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32564c44-e940-47b9-b6de-ef8233fb6bbd None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-32564c44-e940-47b9-b6de-ef8233fb6bbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:38.492989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:38.493895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:38.494738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:38.494738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:38.494738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:38.494738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:38.494738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:38.494738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:38.494738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:38.494738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:38.494738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 306/620] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:38.495398 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32564c44-e940-47b9-b6de-ef8233fb6bbd None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:38.551401 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11410e2-5efa-4842-ad6c-2055b401c7c6 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:38.590431 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ae1af-3ae1-43a8-93f7-6f3e36634c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:38.591098 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ae1af-3ae1-43a8-93f7-6f3e36634c77 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:38.591372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ae1af-3ae1-43a8-93f7-6f3e36634c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:38.591679 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ae1af-3ae1-43a8-93f7-6f3e36634c77 None admin] PATH_INFO: `/v3/services/network` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:38.592227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ae1af-3ae1-43a8-93f7-6f3e36634c77 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:38.593661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ae1af-3ae1-43a8-93f7-6f3e36634c77 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-304ae1af-3ae1-43a8-93f7-6f3e36634c77 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:38.602680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:38.603663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:38.608222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:38.608222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:38.608222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:38.608222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:38.608222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:38.608222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:38.608222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:38.608222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:38.608222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:38.608222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:38.608222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:38.608222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:38.608222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:12:38.608222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:38.610054 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32564c44-e940-47b9-b6de-ef8233fb6bbd None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:38.610150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 315/621] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:38 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:38.610150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ae1af-3ae1-43a8-93f7-6f3e36634c77 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:38.630155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:38.648812 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:38.658568 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a431f0-d5de-4e23-a047-b4c6c28b02e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:38.659250 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a431f0-d5de-4e23-a047-b4c6c28b02e1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:38.659534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a431f0-d5de-4e23-a047-b4c6c28b02e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:38.659806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a431f0-d5de-4e23-a047-b4c6c28b02e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:38.670967 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a431f0-d5de-4e23-a047-b4c6c28b02e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:38.672841 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a431f0-d5de-4e23-a047-b4c6c28b02e1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-22a431f0-d5de-4e23-a047-b4c6c28b02e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:38.687646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:38.688566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:38.689463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:38.689463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:38.689463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:38.689463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 307/622] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:38.690307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a431f0-d5de-4e23-a047-b4c6c28b02e1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:38.728313 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ae1af-3ae1-43a8-93f7-6f3e36634c77 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:38.763038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582f987-e2c0-4aed-9a1d-6995d0353387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:38.763707 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582f987-e2c0-4aed-9a1d-6995d0353387 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:38.764079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582f987-e2c0-4aed-9a1d-6995d0353387 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:38.764262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582f987-e2c0-4aed-9a1d-6995d0353387 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:38.765513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582f987-e2c0-4aed-9a1d-6995d0353387 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:38.765717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582f987-e2c0-4aed-9a1d-6995d0353387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:38.767271 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582f987-e2c0-4aed-9a1d-6995d0353387 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:38.777478 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 316/623] 10.222.0.18 () {56 vars in 1124 bytes} [Fri Oct 14 02:12:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:38.777478 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582f987-e2c0-4aed-9a1d-6995d0353387 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:38.819350 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a431f0-d5de-4e23-a047-b4c6c28b02e1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:38.854532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:38.859837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011668d-4c9a-46e0-ab12-b31f2704750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:38.861181 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011668d-4c9a-46e0-ab12-b31f2704750b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:38.861633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011668d-4c9a-46e0-ab12-b31f2704750b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:38.862050 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011668d-4c9a-46e0-ab12-b31f2704750b None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:38.862745 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011668d-4c9a-46e0-ab12-b31f2704750b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:38.863051 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011668d-4c9a-46e0-ab12-b31f2704750b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:38.864505 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011668d-4c9a-46e0-ab12-b31f2704750b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:38.881995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 308/624] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:38.881995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011668d-4c9a-46e0-ab12-b31f2704750b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:38.884462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582f987-e2c0-4aed-9a1d-6995d0353387 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:38.920337 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb116893-dbaa-4b69-9e04-a95e37ea0199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:38.920981 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb116893-dbaa-4b69-9e04-a95e37ea0199 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:38.921259 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb116893-dbaa-4b69-9e04-a95e37ea0199 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:38.921537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb116893-dbaa-4b69-9e04-a95e37ea0199 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:38.922162 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb116893-dbaa-4b69-9e04-a95e37ea0199 None admin] RBAC: Adding query filter params (name=network) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:38.922468 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb116893-dbaa-4b69-9e04-a95e37ea0199 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:38.923087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:38.924242 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb116893-dbaa-4b69-9e04-a95e37ea0199 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:38.934326 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 317/625] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Oct 14 02:12:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:38.934326 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb116893-dbaa-4b69-9e04-a95e37ea0199 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:38.990908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011668d-4c9a-46e0-ab12-b31f2704750b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:39.027381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71d619-ba07-4866-b611-c49c800994b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:39.028277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d619-ba07-4866-b611-c49c800994b2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:39.028716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d619-ba07-4866-b611-c49c800994b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:39.028995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d619-ba07-4866-b611-c49c800994b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:39.029650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71d619-ba07-4866-b611-c49c800994b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:39.029958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71d619-ba07-4866-b611-c49c800994b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:39.031899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71d619-ba07-4866-b611-c49c800994b2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:39.037429 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb116893-dbaa-4b69-9e04-a95e37ea0199 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:39.043648 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 309/626] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:39.044471 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71d619-ba07-4866-b611-c49c800994b2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:39.075978 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6139628-69ce-4cfa-83a1-ca9bd4192812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:39.076664 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6139628-69ce-4cfa-83a1-ca9bd4192812 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:39.076944 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6139628-69ce-4cfa-83a1-ca9bd4192812 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:39.077212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6139628-69ce-4cfa-83a1-ca9bd4192812 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:39.077840 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6139628-69ce-4cfa-83a1-ca9bd4192812 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:39.078148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6139628-69ce-4cfa-83a1-ca9bd4192812 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:39.079588 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6139628-69ce-4cfa-83a1-ca9bd4192812 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:39.084598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 318/627] 10.222.0.18 () {56 vars in 1124 bytes} [Fri Oct 14 02:12:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:39.085412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6139628-69ce-4cfa-83a1-ca9bd4192812 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:39.089652 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:39.157690 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:39.160739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71d619-ba07-4866-b611-c49c800994b2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:39.192370 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6139628-69ce-4cfa-83a1-ca9bd4192812 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:39.199258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f430dc-bb6e-4793-b587-2ddbc25fa08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:39.200386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f430dc-bb6e-4793-b587-2ddbc25fa08d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:39.200726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f430dc-bb6e-4793-b587-2ddbc25fa08d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:39.201001 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f430dc-bb6e-4793-b587-2ddbc25fa08d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:39.202424 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f430dc-bb6e-4793-b587-2ddbc25fa08d None admin] RBAC: Adding query filter params (role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b, scope.project.id=f28d337589024cc88184355bd0ee8572, user.id=cf0efd6d12ad458388e0c3574bbedc6a) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:39.202740 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f430dc-bb6e-4793-b587-2ddbc25fa08d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:39.204907 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f430dc-bb6e-4793-b587-2ddbc25fa08d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:39.219446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 310/628] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:12:39 2022] GET /identity/v3/role_assignments?user.id=cf0efd6d12ad458388e0c3574bbedc6a&scope.project.id=f28d337589024cc88184355bd0ee8572&role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:39.219446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f430dc-bb6e-4793-b587-2ddbc25fa08d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:39.232963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac2724-3520-45fd-aef8-75f240942a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:39.233792 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac2724-3520-45fd-aef8-75f240942a89 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:39.234135 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac2724-3520-45fd-aef8-75f240942a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:39.234577 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac2724-3520-45fd-aef8-75f240942a89 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:39.235367 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac2724-3520-45fd-aef8-75f240942a89 None admin] RBAC: Adding query filter params (type=network) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:39.235752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac2724-3520-45fd-aef8-75f240942a89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:39.237298 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac2724-3520-45fd-aef8-75f240942a89 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:39.248347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 319/629] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Oct 14 02:12:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:39.249524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac2724-3520-45fd-aef8-75f240942a89 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:39.336122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f430dc-bb6e-4793-b587-2ddbc25fa08d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:39.342519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:39.375001 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc8751-ee15-4d19-80ed-35013fddbe70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:39.376099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc8751-ee15-4d19-80ed-35013fddbe70 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:39.376099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc8751-ee15-4d19-80ed-35013fddbe70 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:39.376461 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc8751-ee15-4d19-80ed-35013fddbe70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:39.384167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc8751-ee15-4d19-80ed-35013fddbe70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:39.385726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc8751-ee15-4d19-80ed-35013fddbe70 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:39.391289 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac2724-3520-45fd-aef8-75f240942a89 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-aebc8751-ee15-4d19-80ed-35013fddbe70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:39.399833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:39.400697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:39.401553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:39.401553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:39.401553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:39.401553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:39.401553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:39.401553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:39.401553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:39.401553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:39.401553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 311/630] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:39.402466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc8751-ee15-4d19-80ed-35013fddbe70 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:39.435441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:39.439360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008976b-d4d9-47e3-8024-2498f64064c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:39.440055 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008976b-d4d9-47e3-8024-2498f64064c9 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:39.440402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008976b-d4d9-47e3-8024-2498f64064c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:39.440738 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008976b-d4d9-47e3-8024-2498f64064c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:39.441480 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008976b-d4d9-47e3-8024-2498f64064c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:39.442912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008976b-d4d9-47e3-8024-2498f64064c9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:39.474257 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 320/631] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:12:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:39.475110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008976b-d4d9-47e3-8024-2498f64064c9 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:39.595791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc8751-ee15-4d19-80ed-35013fddbe70 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:39.680147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e06b19-9796-481b-833b-396679ad0b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:39.680777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e06b19-9796-481b-833b-396679ad0b7f None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:39.681060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e06b19-9796-481b-833b-396679ad0b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:39.681337 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e06b19-9796-481b-833b-396679ad0b7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:39.682025 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e06b19-9796-481b-833b-396679ad0b7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:39.683473 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e06b19-9796-481b-833b-396679ad0b7f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:39.697379 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008976b-d4d9-47e3-8024-2498f64064c9 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:39.722331 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 312/632] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:12:39 2022] POST /identity/v3/endpoints => generated 337 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:39.737661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:39.766176 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01025f3c-5508-4f77-b065-3604998ed1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:39.766845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01025f3c-5508-4f77-b065-3604998ed1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:39.767196 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01025f3c-5508-4f77-b065-3604998ed1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:39.767547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01025f3c-5508-4f77-b065-3604998ed1a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:39.768327 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01025f3c-5508-4f77-b065-3604998ed1a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:39.768704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01025f3c-5508-4f77-b065-3604998ed1a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:39.770329 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01025f3c-5508-4f77-b065-3604998ed1a3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:39.785333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 321/633] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:39.791214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e06b19-9796-481b-833b-396679ad0b7f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:39.889139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:39.925319 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e06b19-9796-481b-833b-396679ad0b7f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:39.962910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf160a5e-19b9-46b1-92e9-113dc4e4f373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:39.963898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf160a5e-19b9-46b1-92e9-113dc4e4f373 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:39.964369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf160a5e-19b9-46b1-92e9-113dc4e4f373 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:39.964852 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf160a5e-19b9-46b1-92e9-113dc4e4f373 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:39.965779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf160a5e-19b9-46b1-92e9-113dc4e4f373 None admin] RBAC: Adding query filter params (role.id=8adf2afafb0446a6992e9aeb380f645d, scope.project.id=f28d337589024cc88184355bd0ee8572, user.id=de87b424a83f48f09fd6c89c5f5621ed) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:39.966298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf160a5e-19b9-46b1-92e9-113dc4e4f373 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:39.968705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf160a5e-19b9-46b1-92e9-113dc4e4f373 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:39.976604 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 313/634] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:12:39 2022] GET /identity/v3/role_assignments?user.id=de87b424a83f48f09fd6c89c5f5621ed&scope.project.id=f28d337589024cc88184355bd0ee8572&role.id=8adf2afafb0446a6992e9aeb380f645d => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:40.521415 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f403-ba86-4c84-8f26-537483092069 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:40.521801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f403-ba86-4c84-8f26-537483092069 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:40.522006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f403-ba86-4c84-8f26-537483092069 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:40.523587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 322/635] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:40.528783 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ba261-846a-4a9a-94c1-39d04fd57f3a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:40.529218 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ba261-846a-4a9a-94c1-39d04fd57f3a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:40.529397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ba261-846a-4a9a-94c1-39d04fd57f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:40.549309 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-058ba261-846a-4a9a-94c1-39d04fd57f3a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:40.551042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-058ba261-846a-4a9a-94c1-39d04fd57f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:40.621610 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 314/636] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:40.899295 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0f403-ba86-4c84-8f26-537483092069 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:41.012497 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0f403-ba86-4c84-8f26-537483092069 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:41.049769 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b24c0-fe4e-4d4c-bc40-8bcc5d76b64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:41.050745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b24c0-fe4e-4d4c-bc40-8bcc5d76b64f None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:41.051260 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b24c0-fe4e-4d4c-bc40-8bcc5d76b64f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:41.051736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b24c0-fe4e-4d4c-bc40-8bcc5d76b64f None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:41.052612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b24c0-fe4e-4d4c-bc40-8bcc5d76b64f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:41.054165 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b24c0-fe4e-4d4c-bc40-8bcc5d76b64f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:41.071173 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 323/637] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Oct 14 02:12:40 2022] POST /identity/v3/services => generated 269 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:41.566157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3dcd1-e3d7-401f-8c9a-fd48631a98af None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:41.566691 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3dcd1-e3d7-401f-8c9a-fd48631a98af None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:41.566691 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3dcd1-e3d7-401f-8c9a-fd48631a98af None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:41.568355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 315/638] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:41.573846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761dbf0-7389-45a1-bd42-6ef752e97f28 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:41.574139 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761dbf0-7389-45a1-bd42-6ef752e97f28 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:41.574414 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761dbf0-7389-45a1-bd42-6ef752e97f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:41.593610 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3761dbf0-7389-45a1-bd42-6ef752e97f28 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:41.594801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3761dbf0-7389-45a1-bd42-6ef752e97f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:41.630950 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:41.674422 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 324/639] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:41 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:41.950735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3dcd1-e3d7-401f-8c9a-fd48631a98af None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:42.017560 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b33363-c5e6-4873-8315-0dbfcdc8b2cc None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:42.018069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b33363-c5e6-4873-8315-0dbfcdc8b2cc None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:42.018144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b33363-c5e6-4873-8315-0dbfcdc8b2cc None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:42.019732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 325/640] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:42.024657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39281f-05c3-4a94-b2ab-d42993238c5a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:42.024939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39281f-05c3-4a94-b2ab-d42993238c5a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:42.025223 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39281f-05c3-4a94-b2ab-d42993238c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:42.029249 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:42.045987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8e39281f-05c3-4a94-b2ab-d42993238c5a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:42.047650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8e39281f-05c3-4a94-b2ab-d42993238c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:42.064945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3dcd1-e3d7-401f-8c9a-fd48631a98af None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:42.107529 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8a2a5-20f5-4a46-9fc1-bc12e31ff7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:42.108283 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8a2a5-20f5-4a46-9fc1-bc12e31ff7ff None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:42.108568 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8a2a5-20f5-4a46-9fc1-bc12e31ff7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:42.108843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8a2a5-20f5-4a46-9fc1-bc12e31ff7ff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:42.109433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8a2a5-20f5-4a46-9fc1-bc12e31ff7ff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:42.110794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8a2a5-20f5-4a46-9fc1-bc12e31ff7ff None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-ccf8a2a5-20f5-4a46-9fc1-bc12e31ff7ff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:42.121018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:42.122274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:42.123119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 316/641] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Oct 14 02:12:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:42.130749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8a2a5-20f5-4a46-9fc1-bc12e31ff7ff None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:42.140632 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 326/642] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:42 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:12:42.248211 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8a2a5-20f5-4a46-9fc1-bc12e31ff7ff None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:42.285366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9076e4-fd65-482e-a2b4-d461697372bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:42.286356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9076e4-fd65-482e-a2b4-d461697372bf None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:42.286839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9076e4-fd65-482e-a2b4-d461697372bf None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:42.287348 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9076e4-fd65-482e-a2b4-d461697372bf None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:42.288180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9076e4-fd65-482e-a2b4-d461697372bf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:42.288686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9076e4-fd65-482e-a2b4-d461697372bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:42.290431 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9076e4-fd65-482e-a2b4-d461697372bf None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:42.291829 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:42.297328 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 317/643] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Oct 14 02:12:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:42.303398 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39281f-05c3-4a94-b2ab-d42993238c5a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:42.324744 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d399eef-e232-434b-9652-0e9a5721a7f0 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:42.325067 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d399eef-e232-434b-9652-0e9a5721a7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:42.325299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d399eef-e232-434b-9652-0e9a5721a7f0 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:42.329059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 318/644] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:12:42.333676 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85328f3f-4a3f-4933-b625-88beea5ab070 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:42.333980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85328f3f-4a3f-4933-b625-88beea5ab070 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:42.334259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85328f3f-4a3f-4933-b625-88beea5ab070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:42.353529 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-85328f3f-4a3f-4933-b625-88beea5ab070 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:42.355248 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-85328f3f-4a3f-4933-b625-88beea5ab070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:42.424309 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:42.431402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39281f-05c3-4a94-b2ab-d42993238c5a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:42.432244 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 319/645] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:42 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:42.439546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85328f3f-4a3f-4933-b625-88beea5ab070 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:42.475240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a85a5-2a6a-4508-9df6-7f7d7d8bc454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:42.476714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a85a5-2a6a-4508-9df6-7f7d7d8bc454 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:42.477278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a85a5-2a6a-4508-9df6-7f7d7d8bc454 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:42.477732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a85a5-2a6a-4508-9df6-7f7d7d8bc454 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:42.478807 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a85a5-2a6a-4508-9df6-7f7d7d8bc454 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:42.479583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a85a5-2a6a-4508-9df6-7f7d7d8bc454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:42.481361 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a85a5-2a6a-4508-9df6-7f7d7d8bc454 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:42.488275 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 327/646] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Oct 14 02:12:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:42.560102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85328f3f-4a3f-4933-b625-88beea5ab070 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:42.591476 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:42.601129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4e384-f263-477c-8b46-618ec0d78625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:42.602079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4e384-f263-477c-8b46-618ec0d78625 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:42.602600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4e384-f263-477c-8b46-618ec0d78625 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:42.603050 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4e384-f263-477c-8b46-618ec0d78625 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:42.603843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4e384-f263-477c-8b46-618ec0d78625 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:42.605447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4e384-f263-477c-8b46-618ec0d78625 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-cef4e384-f263-477c-8b46-618ec0d78625 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:42.616213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:42.617269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:42.618100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:42.618100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:42.618100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:42.618100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:42.618100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:42.618100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:42.618100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:42.618100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:42.618100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:42.618100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:42.618100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:42.618100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:42.618100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:12:42.618100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:42.620583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 320/647] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:12:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:42.627467 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a85a5-2a6a-4508-9df6-7f7d7d8bc454 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:42.704779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4e384-f263-477c-8b46-618ec0d78625 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:42.759704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a85a5-2a6a-4508-9df6-7f7d7d8bc454 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:42.800873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f840d-ba0e-4561-8a9c-931859fa500c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:42.801497 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f840d-ba0e-4561-8a9c-931859fa500c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:42.801772 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f840d-ba0e-4561-8a9c-931859fa500c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:42.802041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f840d-ba0e-4561-8a9c-931859fa500c None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:42.802668 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f840d-ba0e-4561-8a9c-931859fa500c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:42.802967 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f840d-ba0e-4561-8a9c-931859fa500c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:42.804561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f840d-ba0e-4561-8a9c-931859fa500c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:42.813068 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 328/648] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:12:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:42.819467 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f840d-ba0e-4561-8a9c-931859fa500c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:42.846630 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4e384-f263-477c-8b46-618ec0d78625 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:42.857131 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:42.884767 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89e0d2-0b7f-4727-860f-045c570eb237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:42.885320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89e0d2-0b7f-4727-860f-045c570eb237 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:42.885669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89e0d2-0b7f-4727-860f-045c570eb237 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:42.886006 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89e0d2-0b7f-4727-860f-045c570eb237 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:42.897466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89e0d2-0b7f-4727-860f-045c570eb237 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-5f89e0d2-0b7f-4727-860f-045c570eb237 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:42.902914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:42.903835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:42.904667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:42.904667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:42.904667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:42.904667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:42.904667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:42.904667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:42.904667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:42.904667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:42.904667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:42.904667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:42.904667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:12:42.904667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:42.909467 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 321/649] 10.222.0.18 () {56 vars in 1120 bytes} [Fri Oct 14 02:12:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:42.911305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89e0d2-0b7f-4727-860f-045c570eb237 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:42.963620 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f840d-ba0e-4561-8a9c-931859fa500c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:42.968422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:43.012311 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b5868-0bbf-4a28-95f2-9a5d8527b1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:43.013348 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b5868-0bbf-4a28-95f2-9a5d8527b1fb None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:43.013899 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b5868-0bbf-4a28-95f2-9a5d8527b1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:43.014183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b5868-0bbf-4a28-95f2-9a5d8527b1fb None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:43.015842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b5868-0bbf-4a28-95f2-9a5d8527b1fb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:43.016156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b5868-0bbf-4a28-95f2-9a5d8527b1fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:43.018265 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b5868-0bbf-4a28-95f2-9a5d8527b1fb None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:43.022709 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89e0d2-0b7f-4727-860f-045c570eb237 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:43.024363 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 329/650] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:12:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:43.062924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180451d-0ecd-4061-9c98-956f8082aad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:43.063657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180451d-0ecd-4061-9c98-956f8082aad4 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:43.063879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180451d-0ecd-4061-9c98-956f8082aad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:43.064151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180451d-0ecd-4061-9c98-956f8082aad4 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:43.064831 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180451d-0ecd-4061-9c98-956f8082aad4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:43.065134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180451d-0ecd-4061-9c98-956f8082aad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:43.066528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180451d-0ecd-4061-9c98-956f8082aad4 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:43.076091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 322/651] 10.222.0.18 () {56 vars in 1115 bytes} [Fri Oct 14 02:12:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:43.082656 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b5868-0bbf-4a28-95f2-9a5d8527b1fb None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:43.189074 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:43.204184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b5868-0bbf-4a28-95f2-9a5d8527b1fb None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:43.240216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a048f-f159-4601-8383-250a0e1a68e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:43.240857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a048f-f159-4601-8383-250a0e1a68e3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:43.241140 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a048f-f159-4601-8383-250a0e1a68e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:43.241413 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a048f-f159-4601-8383-250a0e1a68e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:43.253072 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a048f-f159-4601-8383-250a0e1a68e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:43.254538 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a048f-f159-4601-8383-250a0e1a68e3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-449a048f-f159-4601-8383-250a0e1a68e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:43.269805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:43.271745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:43.273681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:43.273681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:43.273681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:43.273681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 330/652] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:43.276714 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180451d-0ecd-4061-9c98-956f8082aad4 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:43.309034 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7ce33-e9d7-4622-bd0a-97a4b538a3bc None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:43.309369 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7ce33-e9d7-4622-bd0a-97a4b538a3bc None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:43.309578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7ce33-e9d7-4622-bd0a-97a4b538a3bc None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:43.312403 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 331/653] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:43.317635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebfe36-c8a7-4541-bb18-9b820b1ed139 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:43.317853 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebfe36-c8a7-4541-bb18-9b820b1ed139 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:43.318117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebfe36-c8a7-4541-bb18-9b820b1ed139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:43.344509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c8ebfe36-c8a7-4541-bb18-9b820b1ed139 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:43.346659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c8ebfe36-c8a7-4541-bb18-9b820b1ed139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:43.389599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180451d-0ecd-4061-9c98-956f8082aad4 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:43.405716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:43.420864 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 332/654] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:43 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:12:43.432526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb2963-fa8e-4938-a4cc-270ce5dc3003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:43.433387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb2963-fa8e-4938-a4cc-270ce5dc3003 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:43.433923 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb2963-fa8e-4938-a4cc-270ce5dc3003 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:43.434345 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb2963-fa8e-4938-a4cc-270ce5dc3003 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:43.435171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb2963-fa8e-4938-a4cc-270ce5dc3003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:43.435622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb2963-fa8e-4938-a4cc-270ce5dc3003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:43.437928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb2963-fa8e-4938-a4cc-270ce5dc3003 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:43.451061 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 323/655] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:43.456740 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebfe36-c8a7-4541-bb18-9b820b1ed139 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:43.526638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:43.559244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebfe36-c8a7-4541-bb18-9b820b1ed139 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:43.594940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6abaea-a922-4081-9a00-b1c090ef6e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:43.595528 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6abaea-a922-4081-9a00-b1c090ef6e97 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:43.595814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6abaea-a922-4081-9a00-b1c090ef6e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:43.596087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6abaea-a922-4081-9a00-b1c090ef6e97 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:43.603365 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6abaea-a922-4081-9a00-b1c090ef6e97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:43.604802 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6abaea-a922-4081-9a00-b1c090ef6e97 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-8f6abaea-a922-4081-9a00-b1c090ef6e97 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:43.616576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:43.617490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:43.618596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:43.618596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:43.618596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:43.618596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:43.618596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:43.618596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:43.618596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:43.618596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:43.618596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:12:43.618596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:43.618596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 333/656] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Oct 14 02:12:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:43.626206 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb2963-fa8e-4938-a4cc-270ce5dc3003 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:43.719894 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6abaea-a922-4081-9a00-b1c090ef6e97 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:43.737632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb2963-fa8e-4938-a4cc-270ce5dc3003 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:43.775764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4c4cc-427f-45c4-9a87-8a7bb10c99d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:43.776390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4c4cc-427f-45c4-9a87-8a7bb10c99d8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:43.776674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4c4cc-427f-45c4-9a87-8a7bb10c99d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:43.776950 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4c4cc-427f-45c4-9a87-8a7bb10c99d8 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:43.777687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4c4cc-427f-45c4-9a87-8a7bb10c99d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:43.777996 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4c4cc-427f-45c4-9a87-8a7bb10c99d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:43.779471 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4c4cc-427f-45c4-9a87-8a7bb10c99d8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:43.803114 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 324/657] 10.222.0.18 () {56 vars in 1149 bytes} [Fri Oct 14 02:12:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:43.805762 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4c4cc-427f-45c4-9a87-8a7bb10c99d8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:43.845627 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:43.855852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6abaea-a922-4081-9a00-b1c090ef6e97 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:43.895476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:43.899725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70b91c-d722-4ae2-842e-dd9db72d7c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:43.900299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70b91c-d722-4ae2-842e-dd9db72d7c1d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:43.900614 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70b91c-d722-4ae2-842e-dd9db72d7c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:43.900895 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70b91c-d722-4ae2-842e-dd9db72d7c1d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:43.901609 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70b91c-d722-4ae2-842e-dd9db72d7c1d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:43.903206 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70b91c-d722-4ae2-842e-dd9db72d7c1d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-1b70b91c-d722-4ae2-842e-dd9db72d7c1d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:43.913069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:43.914105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:43.914977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 334/658] 10.222.0.18 () {56 vars in 1132 bytes} [Fri Oct 14 02:12:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:43.923969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70b91c-d722-4ae2-842e-dd9db72d7c1d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:43.928450 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4c4cc-427f-45c4-9a87-8a7bb10c99d8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:43.967408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7dbc7-20f4-4c8f-bd0f-49a16152d1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:43.968236 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7dbc7-20f4-4c8f-bd0f-49a16152d1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:43.968523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7dbc7-20f4-4c8f-bd0f-49a16152d1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:43.968846 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7dbc7-20f4-4c8f-bd0f-49a16152d1a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:43.980914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7dbc7-20f4-4c8f-bd0f-49a16152d1a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:43.983675 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7dbc7-20f4-4c8f-bd0f-49a16152d1a4 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-2cf7dbc7-20f4-4c8f-bd0f-49a16152d1a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:44.001344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:44.002287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:44.003183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:44.003183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:44.003183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:44.007254 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 325/659] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:44.009130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7dbc7-20f4-4c8f-bd0f-49a16152d1a4 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:44.057218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70b91c-d722-4ae2-842e-dd9db72d7c1d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:44.093591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeaa7e7-2266-46e6-b565-e19413292b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:44.094853 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeaa7e7-2266-46e6-b565-e19413292b5e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:44.094853 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeaa7e7-2266-46e6-b565-e19413292b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:44.094853 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeaa7e7-2266-46e6-b565-e19413292b5e None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:44.095498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeaa7e7-2266-46e6-b565-e19413292b5e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:44.095853 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeaa7e7-2266-46e6-b565-e19413292b5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:44.097160 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeaa7e7-2266-46e6-b565-e19413292b5e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:44.106637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 335/660] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Oct 14 02:12:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:44.108541 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeaa7e7-2266-46e6-b565-e19413292b5e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:44.111606 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:44.130889 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7dbc7-20f4-4c8f-bd0f-49a16152d1a4 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:44.168701 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8487f92-ebac-46e4-9c84-68298c103dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:44.169343 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8487f92-ebac-46e4-9c84-68298c103dae None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:44.169622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8487f92-ebac-46e4-9c84-68298c103dae None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:44.169907 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8487f92-ebac-46e4-9c84-68298c103dae None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:44.170937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8487f92-ebac-46e4-9c84-68298c103dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:44.171343 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8487f92-ebac-46e4-9c84-68298c103dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:44.173428 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8487f92-ebac-46e4-9c84-68298c103dae None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:44.191465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:44.194557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 326/661] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:44.196494 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8487f92-ebac-46e4-9c84-68298c103dae None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:44.235812 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeaa7e7-2266-46e6-b565-e19413292b5e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:44.274395 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba3eea-8716-4bcd-9585-42b0aab8ae49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:44.275015 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba3eea-8716-4bcd-9585-42b0aab8ae49 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:44.275323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba3eea-8716-4bcd-9585-42b0aab8ae49 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:44.275597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba3eea-8716-4bcd-9585-42b0aab8ae49 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:44.276232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba3eea-8716-4bcd-9585-42b0aab8ae49 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:44.276535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba3eea-8716-4bcd-9585-42b0aab8ae49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:44.277907 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba3eea-8716-4bcd-9585-42b0aab8ae49 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:44.288356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 336/662] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Oct 14 02:12:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:44.289255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba3eea-8716-4bcd-9585-42b0aab8ae49 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:44.321301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8487f92-ebac-46e4-9c84-68298c103dae None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:44.366816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67059be4-a594-4cf7-ae83-815592379784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:44.367409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67059be4-a594-4cf7-ae83-815592379784 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:44.368011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67059be4-a594-4cf7-ae83-815592379784 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:44.368292 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67059be4-a594-4cf7-ae83-815592379784 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:44.377773 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67059be4-a594-4cf7-ae83-815592379784 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:44.379372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67059be4-a594-4cf7-ae83-815592379784 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-67059be4-a594-4cf7-ae83-815592379784 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:44.391419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:44.392330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:44.393277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:44.393277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:44.393277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:44.393277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:44.393277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:44.393277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:44.393277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:44.393277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:44.393277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 327/663] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:44.396277 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba3eea-8716-4bcd-9585-42b0aab8ae49 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:44.398711 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67059be4-a594-4cf7-ae83-815592379784 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:44.404117 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:44.425490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:44.439281 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb9bec-0d74-490d-acde-66908666dec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:44.439948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb9bec-0d74-490d-acde-66908666dec8 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:44.440286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb9bec-0d74-490d-acde-66908666dec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:44.440568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb9bec-0d74-490d-acde-66908666dec8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:44.441336 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb9bec-0d74-490d-acde-66908666dec8 None admin] RBAC: Adding query filter params (interface=public, service_id=07f91d97970b4db59770cb785ca111ba, region_id=RegionOne) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:44.441653 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb9bec-0d74-490d-acde-66908666dec8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:44.443052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb9bec-0d74-490d-acde-66908666dec8 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:44.453309 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 337/664] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Oct 14 02:12:44 2022] GET /identity/v3/endpoints?service_id=07f91d97970b4db59770cb785ca111ba&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:44.454204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb9bec-0d74-490d-acde-66908666dec8 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:44.544455 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67059be4-a594-4cf7-ae83-815592379784 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:44.559423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb9bec-0d74-490d-acde-66908666dec8 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:44.591090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba961509-132b-43a7-add2-c1882927212b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:44.592076 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba961509-132b-43a7-add2-c1882927212b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:44.593186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba961509-132b-43a7-add2-c1882927212b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:44.593717 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba961509-132b-43a7-add2-c1882927212b None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:44.594588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba961509-132b-43a7-add2-c1882927212b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:44.595194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba961509-132b-43a7-add2-c1882927212b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:44.595693 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dee12c-4f0d-4996-90a6-2a5fce78a016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:44.596373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dee12c-4f0d-4996-90a6-2a5fce78a016 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:44.596721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dee12c-4f0d-4996-90a6-2a5fce78a016 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:44.597064 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dee12c-4f0d-4996-90a6-2a5fce78a016 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:44.597725 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba961509-132b-43a7-add2-c1882927212b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:44.597858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dee12c-4f0d-4996-90a6-2a5fce78a016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:44.599320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dee12c-4f0d-4996-90a6-2a5fce78a016 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:44.611482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 338/665] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Oct 14 02:12:44 2022] GET /identity/v3/services => generated 1029 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:12:44.617693 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 328/666] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:44.619429 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dee12c-4f0d-4996-90a6-2a5fce78a016 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:44.684150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:44.760896 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dee12c-4f0d-4996-90a6-2a5fce78a016 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:44.800960 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b4e77-4ce4-49c0-9155-a96ef5242fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:44.801931 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b4e77-4ce4-49c0-9155-a96ef5242fbb None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:44.802425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b4e77-4ce4-49c0-9155-a96ef5242fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:44.802881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b4e77-4ce4-49c0-9155-a96ef5242fbb None admin] PATH_INFO: `/v3/projects/f28d337589024cc88184355bd0ee8572/users/de87b424a83f48f09fd6c89c5f5621ed/roles/79e6c6fbf4b446da89c0e323be7e232b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:44.805261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b4e77-4ce4-49c0-9155-a96ef5242fbb None admin] RBAC: Authorizing `identity:create_grant(project_id=f28d337589024cc88184355bd0ee8572, user_id=de87b424a83f48f09fd6c89c5f5621ed, role_id=79e6c6fbf4b446da89c0e323be7e232b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:44.806911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b4e77-4ce4-49c0-9155-a96ef5242fbb None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:44.820384 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a0676-dac8-4178-9838-c91d1a01a97a None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:44.821328 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a0676-dac8-4178-9838-c91d1a01a97a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:44.821608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a0676-dac8-4178-9838-c91d1a01a97a None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:44.823103 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 329/667] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:12:44.829520 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42738d8d-c727-4a5e-8a5e-75d7f32e81da None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:44.829809 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42738d8d-c727-4a5e-8a5e-75d7f32e81da None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:44.830074 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42738d8d-c727-4a5e-8a5e-75d7f32e81da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:44.834937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 339/668] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:12:44 2022] PUT /identity/v3/projects/f28d337589024cc88184355bd0ee8572/users/de87b424a83f48f09fd6c89c5f5621ed/roles/79e6c6fbf4b446da89c0e323be7e232b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:12:44.852201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-42738d8d-c727-4a5e-8a5e-75d7f32e81da None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:44.853531 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-42738d8d-c727-4a5e-8a5e-75d7f32e81da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:44.923038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:44.932575 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 330/669] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:44 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:12:45.208489 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b4e77-4ce4-49c0-9155-a96ef5242fbb None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:45.262979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50062-1715-4932-9882-35da5a55ad41 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:45.263447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50062-1715-4932-9882-35da5a55ad41 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:45.263817 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50062-1715-4932-9882-35da5a55ad41 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:45.266216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 331/670] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:45.271590 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf1bc1-f1cb-40f5-9dbd-636b1d9c205e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:45.271929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf1bc1-f1cb-40f5-9dbd-636b1d9c205e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:45.272307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf1bc1-f1cb-40f5-9dbd-636b1d9c205e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:45.292470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-88cf1bc1-f1cb-40f5-9dbd-636b1d9c205e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:45.294216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-88cf1bc1-f1cb-40f5-9dbd-636b1d9c205e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:45.317984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b4e77-4ce4-49c0-9155-a96ef5242fbb None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:45.330299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:45.357770 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b079370-9779-4024-9224-b302ddff909f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:45.359255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b079370-9779-4024-9224-b302ddff909f None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:45.359511 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b079370-9779-4024-9224-b302ddff909f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:45.359754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b079370-9779-4024-9224-b302ddff909f None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:45.360530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b079370-9779-4024-9224-b302ddff909f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:45.361905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b079370-9779-4024-9224-b302ddff909f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:45.387252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 340/671] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Oct 14 02:12:45 2022] POST /identity/v3/services => generated 241 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:45.394528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 332/672] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:45 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:12:45.673247 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b079370-9779-4024-9224-b302ddff909f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:45.791177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b079370-9779-4024-9224-b302ddff909f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:45.826039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6789263-dea8-444e-af7b-ee6d808c3c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:45.826704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6789263-dea8-444e-af7b-ee6d808c3c13 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:45.827207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6789263-dea8-444e-af7b-ee6d808c3c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:45.827530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6789263-dea8-444e-af7b-ee6d808c3c13 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:45.828197 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6789263-dea8-444e-af7b-ee6d808c3c13 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:45.829557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6789263-dea8-444e-af7b-ee6d808c3c13 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:45.850814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 341/673] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Oct 14 02:12:45 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:46.896108 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac09f8f-da89-45a6-a4a6-7cfb2eafe9e9 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:46.896650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac09f8f-da89-45a6-a4a6-7cfb2eafe9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:46.896650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac09f8f-da89-45a6-a4a6-7cfb2eafe9e9 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:46.898196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 333/674] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:46.903521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686028a3-88a8-4abb-92b1-32fc79f43095 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:46.904343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686028a3-88a8-4abb-92b1-32fc79f43095 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:46.904908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686028a3-88a8-4abb-92b1-32fc79f43095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:46.924033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-686028a3-88a8-4abb-92b1-32fc79f43095 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:46.925485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-686028a3-88a8-4abb-92b1-32fc79f43095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:46.953441 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:47.006345 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 342/675] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:47.059024 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066fb29-583e-472d-8804-5c271c960773 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:47.059599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066fb29-583e-472d-8804-5c271c960773 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:47.060296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066fb29-583e-472d-8804-5c271c960773 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:47.062195 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 334/676] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:47.067582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9554e-493a-4460-8c85-947d46847988 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:47.067973 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9554e-493a-4460-8c85-947d46847988 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:47.068350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9554e-493a-4460-8c85-947d46847988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:47.087325 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6ab9554e-493a-4460-8c85-947d46847988 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:47.088485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6ab9554e-493a-4460-8c85-947d46847988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:47.157019 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 343/677] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:47.281748 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066fb29-583e-472d-8804-5c271c960773 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:47.384589 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066fb29-583e-472d-8804-5c271c960773 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:47.405173 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:47.423347 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cedb1-1e96-46ea-bde8-07b74a173d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:47.423986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cedb1-1e96-46ea-bde8-07b74a173d07 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:47.424261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cedb1-1e96-46ea-bde8-07b74a173d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:47.424532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cedb1-1e96-46ea-bde8-07b74a173d07 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:47.425112 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3cedb1-1e96-46ea-bde8-07b74a173d07 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:47.426590 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3cedb1-1e96-46ea-bde8-07b74a173d07 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:47.432556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9554e-493a-4460-8c85-947d46847988 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-0a3cedb1-1e96-46ea-bde8-07b74a173d07 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:47.438203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:47.439140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:47.440021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 335/678] 10.222.0.18 () {56 vars in 1132 bytes} [Fri Oct 14 02:12:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:47.445985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cedb1-1e96-46ea-bde8-07b74a173d07 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:47.546665 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:47.568386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cedb1-1e96-46ea-bde8-07b74a173d07 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:47.571327 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9554e-493a-4460-8c85-947d46847988 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:47.606156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f957cb-1531-40d3-95dc-456ca47c26f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:47.606774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f957cb-1531-40d3-95dc-456ca47c26f2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:47.607050 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f957cb-1531-40d3-95dc-456ca47c26f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:47.607347 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f957cb-1531-40d3-95dc-456ca47c26f2 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:47.609054 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f957cb-1531-40d3-95dc-456ca47c26f2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:47.609360 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f957cb-1531-40d3-95dc-456ca47c26f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:47.611193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f957cb-1531-40d3-95dc-456ca47c26f2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:47.616116 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 336/679] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Oct 14 02:12:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:47.617127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f21f0-33fe-4592-855b-556d66f55c9b None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:47.617379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f21f0-33fe-4592-855b-556d66f55c9b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:47.617648 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f21f0-33fe-4592-855b-556d66f55c9b None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:47.617741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c817c0-19fe-406a-8fc2-a4d5d87e04ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:47.618346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c817c0-19fe-406a-8fc2-a4d5d87e04ac None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:47.618617 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c817c0-19fe-406a-8fc2-a4d5d87e04ac None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:47.620109 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c817c0-19fe-406a-8fc2-a4d5d87e04ac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:47.620217 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 337/680] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:47.624261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f21f0-33fe-4592-855b-556d66f55c9b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:47.632661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c817c0-19fe-406a-8fc2-a4d5d87e04ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-14c817c0-19fe-406a-8fc2-a4d5d87e04ac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:12:47.637484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:47.638396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:47.639904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:47.639904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:47.639904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:47.639904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:47.639904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:12:47.639904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:12:47.639904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:12:47.639904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:12:47.639904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:12:47.639904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:12:47.639904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:12:47.639904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:47.644239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 344/681] 10.222.0.18 () {56 vars in 1120 bytes} [Fri Oct 14 02:12:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:47.645322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48881a87-bdb6-4115-ac25-efe64c80e6f4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:47.645612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48881a87-bdb6-4115-ac25-efe64c80e6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:47.645891 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48881a87-bdb6-4115-ac25-efe64c80e6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:47.665249 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-48881a87-bdb6-4115-ac25-efe64c80e6f4 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:47.666438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-48881a87-bdb6-4115-ac25-efe64c80e6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:47.679818 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:47.740832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f21f0-33fe-4592-855b-556d66f55c9b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:47.742506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 345/682] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:47.748165 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48881a87-bdb6-4115-ac25-efe64c80e6f4 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:47.783819 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29c72a-f370-4039-9063-93034f734d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:47.784851 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29c72a-f370-4039-9063-93034f734d00 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:47.785346 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29c72a-f370-4039-9063-93034f734d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:47.785898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29c72a-f370-4039-9063-93034f734d00 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:47.786835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29c72a-f370-4039-9063-93034f734d00 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:47.787363 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29c72a-f370-4039-9063-93034f734d00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:47.788970 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29c72a-f370-4039-9063-93034f734d00 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:47.800567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 338/683] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Oct 14 02:12:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:47.801688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29c72a-f370-4039-9063-93034f734d00 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:47.843005 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:47.880504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48881a87-bdb6-4115-ac25-efe64c80e6f4 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:47.916877 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cbe4b-be04-4668-9000-672fe96740d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:47.917535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cbe4b-be04-4668-9000-672fe96740d5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:47.917768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cbe4b-be04-4668-9000-672fe96740d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:47.918024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cbe4b-be04-4668-9000-672fe96740d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:47.918678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cbe4b-be04-4668-9000-672fe96740d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:47.918982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cbe4b-be04-4668-9000-672fe96740d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:47.920720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cbe4b-be04-4668-9000-672fe96740d5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:47.929274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29c72a-f370-4039-9063-93034f734d00 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:47.932823 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 346/684] 10.222.0.18 () {56 vars in 1115 bytes} [Fri Oct 14 02:12:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:47.939340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cbe4b-be04-4668-9000-672fe96740d5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:47.946295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:47.973417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fda98-4cd2-4ae5-91ef-53395993fc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:47.974043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fda98-4cd2-4ae5-91ef-53395993fc2a None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:47.974323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fda98-4cd2-4ae5-91ef-53395993fc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:47.974596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fda98-4cd2-4ae5-91ef-53395993fc2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:47.975573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fda98-4cd2-4ae5-91ef-53395993fc2a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:47.977764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fda98-4cd2-4ae5-91ef-53395993fc2a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:48.011133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 339/685] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:12:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:48.023253 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fda98-4cd2-4ae5-91ef-53395993fc2a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:48.076669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cbe4b-be04-4668-9000-672fe96740d5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:48.125646 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff55902-a004-48fd-a557-33c881401243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:48.126629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff55902-a004-48fd-a557-33c881401243 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:48.127051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff55902-a004-48fd-a557-33c881401243 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:48.127611 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff55902-a004-48fd-a557-33c881401243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:48.130234 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:48.141861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff55902-a004-48fd-a557-33c881401243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:48.143446 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff55902-a004-48fd-a557-33c881401243 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-1ff55902-a004-48fd-a557-33c881401243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:48.159749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:48.160733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:48.161595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:48.161595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:48.161595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:48.161769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fda98-4cd2-4ae5-91ef-53395993fc2a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:48.161845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 347/686] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:48.162578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7a4a5-0265-4d80-bfbd-66e8e5f5c55d None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:48.162875 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7a4a5-0265-4d80-bfbd-66e8e5f5c55d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:48.163168 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7a4a5-0265-4d80-bfbd-66e8e5f5c55d None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:48.164721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 348/687] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:48.166855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7a4a5-0265-4d80-bfbd-66e8e5f5c55d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:48.209764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fa41b-ae9b-441f-86b7-ebb266186fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:48.210408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fa41b-ae9b-441f-86b7-ebb266186fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:48.210685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fa41b-ae9b-441f-86b7-ebb266186fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:48.210963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fa41b-ae9b-441f-86b7-ebb266186fe6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:48.211642 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395fa41b-ae9b-441f-86b7-ebb266186fe6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:48.213012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395fa41b-ae9b-441f-86b7-ebb266186fe6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-395fa41b-ae9b-441f-86b7-ebb266186fe6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:48.222965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:48.223981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:48.226046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 340/688] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Oct 14 02:12:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:48.226814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d57c9-a6bf-4c15-8a1e-c6b1b33b759b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:48.226814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d57c9-a6bf-4c15-8a1e-c6b1b33b759b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:48.226814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d57c9-a6bf-4c15-8a1e-c6b1b33b759b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:48.245066 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-bd0d57c9-a6bf-4c15-8a1e-c6b1b33b759b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:48.246263 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bd0d57c9-a6bf-4c15-8a1e-c6b1b33b759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:48.275984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:48.288936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7a4a5-0265-4d80-bfbd-66e8e5f5c55d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:48.321005 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 341/689] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:48.321921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d57c9-a6bf-4c15-8a1e-c6b1b33b759b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:48.335918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bf191-9600-497b-aec8-ddf82effd848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:48.337107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bf191-9600-497b-aec8-ddf82effd848 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:48.338051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bf191-9600-497b-aec8-ddf82effd848 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:48.338530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bf191-9600-497b-aec8-ddf82effd848 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:48.339657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bf191-9600-497b-aec8-ddf82effd848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:48.340412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bf191-9600-497b-aec8-ddf82effd848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:48.343254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bf191-9600-497b-aec8-ddf82effd848 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:48.364982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 349/690] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:48.366417 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bf191-9600-497b-aec8-ddf82effd848 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:48.417593 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:48.456029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d57c9-a6bf-4c15-8a1e-c6b1b33b759b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:48.482420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bf191-9600-497b-aec8-ddf82effd848 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:48.492772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2f38c-e05e-4974-aaee-f172ad45609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:48.493525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2f38c-e05e-4974-aaee-f172ad45609b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:48.493710 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2f38c-e05e-4974-aaee-f172ad45609b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:48.493989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2f38c-e05e-4974-aaee-f172ad45609b None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:48.494676 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2f38c-e05e-4974-aaee-f172ad45609b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:48.494982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2f38c-e05e-4974-aaee-f172ad45609b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:48.496447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2f38c-e05e-4974-aaee-f172ad45609b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:48.506618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 342/691] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Oct 14 02:12:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:48.507858 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2f38c-e05e-4974-aaee-f172ad45609b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:48.530216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef999096-1150-42d8-a359-97a81a7ac187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:48.531789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef999096-1150-42d8-a359-97a81a7ac187 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:48.532253 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef999096-1150-42d8-a359-97a81a7ac187 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:48.532831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef999096-1150-42d8-a359-97a81a7ac187 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:48.537706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:48.548287 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef999096-1150-42d8-a359-97a81a7ac187 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:48.550018 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef999096-1150-42d8-a359-97a81a7ac187 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-ef999096-1150-42d8-a359-97a81a7ac187 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:48.565639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:48.566575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:48.567525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:48.567525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:48.567525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:48.567525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:48.567525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:48.567525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:48.567525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:48.567525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:48.567525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:12:48.567525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:48.574045 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 350/692] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Oct 14 02:12:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:48.574995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef999096-1150-42d8-a359-97a81a7ac187 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:48.624462 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2f38c-e05e-4974-aaee-f172ad45609b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:48.660551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf69bf-c89f-4c01-84f4-ef74c224694e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:48.661168 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf69bf-c89f-4c01-84f4-ef74c224694e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:48.661449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf69bf-c89f-4c01-84f4-ef74c224694e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:48.661722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf69bf-c89f-4c01-84f4-ef74c224694e None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:48.662352 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf69bf-c89f-4c01-84f4-ef74c224694e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:48.662766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf69bf-c89f-4c01-84f4-ef74c224694e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:48.664167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf69bf-c89f-4c01-84f4-ef74c224694e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:48.674779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 343/693] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Oct 14 02:12:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:48.674779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf69bf-c89f-4c01-84f4-ef74c224694e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:48.691952 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef999096-1150-42d8-a359-97a81a7ac187 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:48.699024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:48.729320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7976e6-4a3c-4cb5-8f71-0df1b11975f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:48.730613 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7976e6-4a3c-4cb5-8f71-0df1b11975f5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:48.730957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7976e6-4a3c-4cb5-8f71-0df1b11975f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:48.731265 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7976e6-4a3c-4cb5-8f71-0df1b11975f5 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:48.732004 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7976e6-4a3c-4cb5-8f71-0df1b11975f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:48.732310 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7976e6-4a3c-4cb5-8f71-0df1b11975f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:48.734222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7976e6-4a3c-4cb5-8f71-0df1b11975f5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:48.756905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 351/694] 10.222.0.18 () {56 vars in 1149 bytes} [Fri Oct 14 02:12:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:48.757754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7976e6-4a3c-4cb5-8f71-0df1b11975f5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:48.789418 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:48.800857 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf69bf-c89f-4c01-84f4-ef74c224694e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:48.837887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98206325-45c6-4ce0-94df-7ef8ef7f7e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:48.838533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98206325-45c6-4ce0-94df-7ef8ef7f7e0d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:48.838812 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98206325-45c6-4ce0-94df-7ef8ef7f7e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:48.839104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98206325-45c6-4ce0-94df-7ef8ef7f7e0d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:48.839691 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98206325-45c6-4ce0-94df-7ef8ef7f7e0d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:48.841052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98206325-45c6-4ce0-94df-7ef8ef7f7e0d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-98206325-45c6-4ce0-94df-7ef8ef7f7e0d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:48.850454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:48.851493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:48.856381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 344/695] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:12:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:48.857386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98206325-45c6-4ce0-94df-7ef8ef7f7e0d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:48.866507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7976e6-4a3c-4cb5-8f71-0df1b11975f5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:48.903148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6469f-066f-4598-a3bf-985987bc3ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:48.903690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6469f-066f-4598-a3bf-985987bc3ccb None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:48.903977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6469f-066f-4598-a3bf-985987bc3ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:48.904246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6469f-066f-4598-a3bf-985987bc3ccb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:48.905573 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6469f-066f-4598-a3bf-985987bc3ccb None admin] RBAC: Adding query filter params (interface=public, service_id=c547dc458810462a865b9d929a05e795, region_id=RegionOne) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:48.905885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6469f-066f-4598-a3bf-985987bc3ccb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:48.907270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6469f-066f-4598-a3bf-985987bc3ccb None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:48.912514 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 352/696] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Oct 14 02:12:48 2022] GET /identity/v3/endpoints?service_id=c547dc458810462a865b9d929a05e795&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:48.913145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6469f-066f-4598-a3bf-985987bc3ccb None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:48.943818 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:48.973661 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98206325-45c6-4ce0-94df-7ef8ef7f7e0d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:49.010639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21723cf5-4d54-4b9f-a653-b926d6bd0277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:49.011309 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21723cf5-4d54-4b9f-a653-b926d6bd0277 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:49.011588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21723cf5-4d54-4b9f-a653-b926d6bd0277 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:49.011858 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21723cf5-4d54-4b9f-a653-b926d6bd0277 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:49.021571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6469f-066f-4598-a3bf-985987bc3ccb None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:49.037566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:49.058088 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c984ab1-8386-425f-8e9b-0b01f75ef305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:49.058696 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c984ab1-8386-425f-8e9b-0b01f75ef305 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:49.058975 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c984ab1-8386-425f-8e9b-0b01f75ef305 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:49.059273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c984ab1-8386-425f-8e9b-0b01f75ef305 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:49.059907 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c984ab1-8386-425f-8e9b-0b01f75ef305 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:49.060208 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c984ab1-8386-425f-8e9b-0b01f75ef305 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:49.061615 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c984ab1-8386-425f-8e9b-0b01f75ef305 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:49.070684 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 353/697] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:12:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:49.070684 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c984ab1-8386-425f-8e9b-0b01f75ef305 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:49.092705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21723cf5-4d54-4b9f-a653-b926d6bd0277 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:49.094343 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21723cf5-4d54-4b9f-a653-b926d6bd0277 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:49.161064 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:49.173126 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c984ab1-8386-425f-8e9b-0b01f75ef305 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:49.207645 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cf3d2-c4f8-45f3-a432-0393672016c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:49.212610 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 345/698] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:12:48 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:12:49.212744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cf3d2-c4f8-45f3-a432-0393672016c7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:49.212744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cf3d2-c4f8-45f3-a432-0393672016c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:49.212897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21723cf5-4d54-4b9f-a653-b926d6bd0277 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:49.212989 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cf3d2-c4f8-45f3-a432-0393672016c7 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:49.212989 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2cf3d2-c4f8-45f3-a432-0393672016c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:49.212989 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2cf3d2-c4f8-45f3-a432-0393672016c7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:49.218981 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 354/699] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Oct 14 02:12:49 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:12:49.219682 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cf3d2-c4f8-45f3-a432-0393672016c7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:49.299962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:49.345934 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cf3d2-c4f8-45f3-a432-0393672016c7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:49.358379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21723cf5-4d54-4b9f-a653-b926d6bd0277 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:49.388582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00585669-9c27-4f70-a85a-75ad022df367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:49.389624 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00585669-9c27-4f70-a85a-75ad022df367 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:49.390190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00585669-9c27-4f70-a85a-75ad022df367 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:49.390718 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00585669-9c27-4f70-a85a-75ad022df367 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:49.399830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b633533-7b82-4ce7-8308-d09ece352e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:49.400457 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b633533-7b82-4ce7-8308-d09ece352e0b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:49.400738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b633533-7b82-4ce7-8308-d09ece352e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:49.401012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b633533-7b82-4ce7-8308-d09ece352e0b None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:49.401645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b633533-7b82-4ce7-8308-d09ece352e0b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:49.401947 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b633533-7b82-4ce7-8308-d09ece352e0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:49.403359 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b633533-7b82-4ce7-8308-d09ece352e0b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:49.403463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00585669-9c27-4f70-a85a-75ad022df367 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:49.405354 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00585669-9c27-4f70-a85a-75ad022df367 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:49.410464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 346/700] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:12:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:49.417366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b633533-7b82-4ce7-8308-d09ece352e0b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-00585669-9c27-4f70-a85a-75ad022df367 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:12:49.424346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:49.425273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:12:49.426171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:12:49.426171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:12:49.426171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:49.427261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 355/701] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:49.431498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00585669-9c27-4f70-a85a-75ad022df367 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:49.443842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:49.551627 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00585669-9c27-4f70-a85a-75ad022df367 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:49.552231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b633533-7b82-4ce7-8308-d09ece352e0b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:49.579728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:49.599777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f52cd4-a7cf-4a5e-949e-cf3ec3bb14a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:49.600402 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f52cd4-a7cf-4a5e-949e-cf3ec3bb14a7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:49.600686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f52cd4-a7cf-4a5e-949e-cf3ec3bb14a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:49.601013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f52cd4-a7cf-4a5e-949e-cf3ec3bb14a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:49.601729 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f52cd4-a7cf-4a5e-949e-cf3ec3bb14a7 None admin] RBAC: Adding query filter params (interface=public, service_id=562b9becdd244c8fa6015fe63e16e378, region_id=RegionOne) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:49.602108 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f52cd4-a7cf-4a5e-949e-cf3ec3bb14a7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:49.603549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f52cd4-a7cf-4a5e-949e-cf3ec3bb14a7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:49.603749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8756b-17f9-4db6-94d5-6263a6bedba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:49.604506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8756b-17f9-4db6-94d5-6263a6bedba8 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:49.605204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8756b-17f9-4db6-94d5-6263a6bedba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:49.605621 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8756b-17f9-4db6-94d5-6263a6bedba8 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:49.606889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8756b-17f9-4db6-94d5-6263a6bedba8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:49.607702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8756b-17f9-4db6-94d5-6263a6bedba8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:49.609936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8756b-17f9-4db6-94d5-6263a6bedba8 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:49.614022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 347/702] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Oct 14 02:12:49 2022] GET /identity/v3/endpoints?service_id=562b9becdd244c8fa6015fe63e16e378&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:49.614980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f52cd4-a7cf-4a5e-949e-cf3ec3bb14a7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:49.631953 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 356/703] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:49.632874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8756b-17f9-4db6-94d5-6263a6bedba8 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:49.703678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:49.739232 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f52cd4-a7cf-4a5e-949e-cf3ec3bb14a7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:49.742999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8756b-17f9-4db6-94d5-6263a6bedba8 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:49.778728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e7724-31d5-4665-9ea0-c5f9e692526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:49.779704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e7724-31d5-4665-9ea0-c5f9e692526b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:49.779909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e370ee-63a1-4747-85d9-3abb6f80d15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:49.780318 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e7724-31d5-4665-9ea0-c5f9e692526b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:49.780565 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e370ee-63a1-4747-85d9-3abb6f80d15d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:49.780848 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e370ee-63a1-4747-85d9-3abb6f80d15d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:49.781014 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e7724-31d5-4665-9ea0-c5f9e692526b None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:49.781125 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e370ee-63a1-4747-85d9-3abb6f80d15d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:49.781861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e7724-31d5-4665-9ea0-c5f9e692526b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:49.783557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e7724-31d5-4665-9ea0-c5f9e692526b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:49.788240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e370ee-63a1-4747-85d9-3abb6f80d15d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:49.789736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e370ee-63a1-4747-85d9-3abb6f80d15d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:49.790524 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 348/704] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Oct 14 02:12:49 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-97e370ee-63a1-4747-85d9-3abb6f80d15d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:49.807918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:49.809038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:49.809984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:49.809984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:49.809984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:49.809984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:49.809984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:49.809984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:49.809984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:49.809984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:49.812466 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 357/705] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:49.818155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e7724-31d5-4665-9ea0-c5f9e692526b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:49.870453 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:49.935420 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e7724-31d5-4665-9ea0-c5f9e692526b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:49.974149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd65b0-1ec5-4846-bdb3-e1af93dd731b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:49.974774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd65b0-1ec5-4846-bdb3-e1af93dd731b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:49.974985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd65b0-1ec5-4846-bdb3-e1af93dd731b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:49.975242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd65b0-1ec5-4846-bdb3-e1af93dd731b None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:49.975927 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd65b0-1ec5-4846-bdb3-e1af93dd731b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:49.976228 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd65b0-1ec5-4846-bdb3-e1af93dd731b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:49.977642 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd65b0-1ec5-4846-bdb3-e1af93dd731b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:49.990924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 349/706] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:12:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:49.997526 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e370ee-63a1-4747-85d9-3abb6f80d15d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:50.104399 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e370ee-63a1-4747-85d9-3abb6f80d15d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:50.136946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:50.146102 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53968e-75cd-460b-81e4-59727c67ed19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:50.146924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53968e-75cd-460b-81e4-59727c67ed19 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:50.147397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53968e-75cd-460b-81e4-59727c67ed19 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:50.148053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53968e-75cd-460b-81e4-59727c67ed19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:50.149170 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53968e-75cd-460b-81e4-59727c67ed19 None admin] RBAC: Adding query filter params (role.id=79e6c6fbf4b446da89c0e323be7e232b, scope.project.id=f28d337589024cc88184355bd0ee8572, user.id=de87b424a83f48f09fd6c89c5f5621ed) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:50.149870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53968e-75cd-460b-81e4-59727c67ed19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:50.151456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53968e-75cd-460b-81e4-59727c67ed19 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:50.159325 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 358/707] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:12:49 2022] GET /identity/v3/role_assignments?user.id=de87b424a83f48f09fd6c89c5f5621ed&scope.project.id=f28d337589024cc88184355bd0ee8572&role.id=79e6c6fbf4b446da89c0e323be7e232b => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:50.241724 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f451e6-6e80-4deb-b1b3-0320ea61e906 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:50.241921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f451e6-6e80-4deb-b1b3-0320ea61e906 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:50.242178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f451e6-6e80-4deb-b1b3-0320ea61e906 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:50.243863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 350/708] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:50.250320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126f3b-3907-4325-8973-36e46c8ba05a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:50.250603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126f3b-3907-4325-8973-36e46c8ba05a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:50.250883 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126f3b-3907-4325-8973-36e46c8ba05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:50.270980 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6a126f3b-3907-4325-8973-36e46c8ba05a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:50.272216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6a126f3b-3907-4325-8973-36e46c8ba05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:50.351874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 359/709] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:50.619414 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f451e6-6e80-4deb-b1b3-0320ea61e906 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:50.729176 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f451e6-6e80-4deb-b1b3-0320ea61e906 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:50.738395 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:50.770482 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb451db-4908-4368-831e-b59a39ec1dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:50.771266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb451db-4908-4368-831e-b59a39ec1dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:50.771552 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb451db-4908-4368-831e-b59a39ec1dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:50.771875 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb451db-4908-4368-831e-b59a39ec1dd0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:50.772466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb451db-4908-4368-831e-b59a39ec1dd0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:50.773827 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb451db-4908-4368-831e-b59a39ec1dd0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-3bb451db-4908-4368-831e-b59a39ec1dd0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:50.783242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:50.784180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:50.784977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 351/710] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:50.790601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126f3b-3907-4325-8973-36e46c8ba05a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:50.892772 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126f3b-3907-4325-8973-36e46c8ba05a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:50.916423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:50.929428 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad587e5-b16c-44ab-93d6-af1ff112c373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:50.930046 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad587e5-b16c-44ab-93d6-af1ff112c373 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:50.930360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad587e5-b16c-44ab-93d6-af1ff112c373 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:50.930641 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad587e5-b16c-44ab-93d6-af1ff112c373 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:50.931290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad587e5-b16c-44ab-93d6-af1ff112c373 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:50.931591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad587e5-b16c-44ab-93d6-af1ff112c373 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:50.932956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad587e5-b16c-44ab-93d6-af1ff112c373 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:50.938600 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 360/711] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Oct 14 02:12:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:50.944212 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb451db-4908-4368-831e-b59a39ec1dd0 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:51.048590 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb451db-4908-4368-831e-b59a39ec1dd0 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:51.083983 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f45a0-43a2-4970-9ef3-3374c1737f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:51.084601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f45a0-43a2-4970-9ef3-3374c1737f79 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:51.084881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f45a0-43a2-4970-9ef3-3374c1737f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:51.085153 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f45a0-43a2-4970-9ef3-3374c1737f79 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:51.085792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f45a0-43a2-4970-9ef3-3374c1737f79 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:51.086098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f45a0-43a2-4970-9ef3-3374c1737f79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:51.087503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f45a0-43a2-4970-9ef3-3374c1737f79 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:51.093084 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 352/712] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Oct 14 02:12:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:51.517438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2437ee-4c29-4c42-8fff-6d1104f19e20 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:51.517996 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2437ee-4c29-4c42-8fff-6d1104f19e20 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:51.517996 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2437ee-4c29-4c42-8fff-6d1104f19e20 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:51.519517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 361/713] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:51.525149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246b65a-34cb-461a-80da-ca1835e15983 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:51.525660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246b65a-34cb-461a-80da-ca1835e15983 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:51.526152 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246b65a-34cb-461a-80da-ca1835e15983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:51.545571 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9246b65a-34cb-461a-80da-ca1835e15983 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:51.547125 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9246b65a-34cb-461a-80da-ca1835e15983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:51.574161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:51.619243 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 353/714] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:12:51.886685 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2437ee-4c29-4c42-8fff-6d1104f19e20 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:51.997958 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2437ee-4c29-4c42-8fff-6d1104f19e20 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:52.035834 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95354f7-9d1f-4ee6-ae3d-494057f23029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:52.036462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95354f7-9d1f-4ee6-ae3d-494057f23029 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:52.036737 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95354f7-9d1f-4ee6-ae3d-494057f23029 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:52.037004 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95354f7-9d1f-4ee6-ae3d-494057f23029 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:52.037582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95354f7-9d1f-4ee6-ae3d-494057f23029 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:52.038927 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95354f7-9d1f-4ee6-ae3d-494057f23029 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-c95354f7-9d1f-4ee6-ae3d-494057f23029 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:52.048485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:52.049464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:52.050659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 362/715] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Oct 14 02:12:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:52.057779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246b65a-34cb-461a-80da-ca1835e15983 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:52.097936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e96ed-790e-4a85-916d-193a0976e94c None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:52.098248 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e96ed-790e-4a85-916d-193a0976e94c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:52.098493 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e96ed-790e-4a85-916d-193a0976e94c None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:52.100117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 363/716] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:12:52.105068 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08220d-d153-4289-8cea-7dde5fbf0d04 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:52.105353 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08220d-d153-4289-8cea-7dde5fbf0d04 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:52.105630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08220d-d153-4289-8cea-7dde5fbf0d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:52.127339 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0a08220d-d153-4289-8cea-7dde5fbf0d04 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:52.129080 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0a08220d-d153-4289-8cea-7dde5fbf0d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:52.166493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246b65a-34cb-461a-80da-ca1835e15983 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:52.201704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:52.204389 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 364/717] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:12:52.212399 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b792d-5f23-43bb-b6fb-1e12194283ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:52.213234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b792d-5f23-43bb-b6fb-1e12194283ff None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:52.213724 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b792d-5f23-43bb-b6fb-1e12194283ff None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:52.214204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b792d-5f23-43bb-b6fb-1e12194283ff None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:52.215101 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b792d-5f23-43bb-b6fb-1e12194283ff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:52.215578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b792d-5f23-43bb-b6fb-1e12194283ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:52.217265 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b792d-5f23-43bb-b6fb-1e12194283ff None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:52.227182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 354/718] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Oct 14 02:12:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:52.230210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08220d-d153-4289-8cea-7dde5fbf0d04 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:52.336151 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08220d-d153-4289-8cea-7dde5fbf0d04 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:52.371444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fac39-78a8-4e03-a554-54773398261f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:52.372324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fac39-78a8-4e03-a554-54773398261f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:52.372595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fac39-78a8-4e03-a554-54773398261f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:52.372861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fac39-78a8-4e03-a554-54773398261f None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:52.373571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fac39-78a8-4e03-a554-54773398261f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:52.373869 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fac39-78a8-4e03-a554-54773398261f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:52.375324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fac39-78a8-4e03-a554-54773398261f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:52.381445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 365/719] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Oct 14 02:12:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:52.387678 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b792d-5f23-43bb-b6fb-1e12194283ff None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:52.420258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:52.492300 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60f292-fdf2-4c70-91ce-ad25f2dd9ec3 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:52.492651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60f292-fdf2-4c70-91ce-ad25f2dd9ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:52.492953 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60f292-fdf2-4c70-91ce-ad25f2dd9ec3 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:52.494181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 366/720] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:52.497433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b792d-5f23-43bb-b6fb-1e12194283ff None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:52.500496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccb740-dd0d-4fc6-9809-571a7889c086 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:52.500782 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccb740-dd0d-4fc6-9809-571a7889c086 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:52.501054 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccb740-dd0d-4fc6-9809-571a7889c086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:52.521210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a9ccb740-dd0d-4fc6-9809-571a7889c086 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:52.522374 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a9ccb740-dd0d-4fc6-9809-571a7889c086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:52.533994 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5374c6-19f8-4d5d-87c8-4a5c0a310924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:52.534610 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5374c6-19f8-4d5d-87c8-4a5c0a310924 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:52.534894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5374c6-19f8-4d5d-87c8-4a5c0a310924 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:52.535197 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5374c6-19f8-4d5d-87c8-4a5c0a310924 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:52.535873 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5374c6-19f8-4d5d-87c8-4a5c0a310924 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:52.537292 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5374c6-19f8-4d5d-87c8-4a5c0a310924 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:52.558932 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 355/721] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:12:52 2022] POST /identity/v3/endpoints => generated 349 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:52.565773 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:52.568129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5374c6-19f8-4d5d-87c8-4a5c0a310924 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:52.613128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 367/722] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:12:52.688269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:52.707836 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5374c6-19f8-4d5d-87c8-4a5c0a310924 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:52.748335 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8dd22-244e-4180-a670-95857ebdb6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:52.749289 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8dd22-244e-4180-a670-95857ebdb6cd None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:52.749573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8dd22-244e-4180-a670-95857ebdb6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:52.749852 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8dd22-244e-4180-a670-95857ebdb6cd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:52.750443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8dd22-244e-4180-a670-95857ebdb6cd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:52.751856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8dd22-244e-4180-a670-95857ebdb6cd None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-bfa8dd22-244e-4180-a670-95857ebdb6cd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:52.760981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:52.761945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:52.763469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 356/723] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:12:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:52.773018 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccb740-dd0d-4fc6-9809-571a7889c086 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:52.881193 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccb740-dd0d-4fc6-9809-571a7889c086 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:52.898176 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8dd22-244e-4180-a670-95857ebdb6cd None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:52.927025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ecf4e-eb62-4ca3-8311-9f3e5ea76413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:52.928214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ecf4e-eb62-4ca3-8311-9f3e5ea76413 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:52.928667 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ecf4e-eb62-4ca3-8311-9f3e5ea76413 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:52.929371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ecf4e-eb62-4ca3-8311-9f3e5ea76413 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:52.930612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ecf4e-eb62-4ca3-8311-9f3e5ea76413 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:52.931336 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ecf4e-eb62-4ca3-8311-9f3e5ea76413 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:52.933521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ecf4e-eb62-4ca3-8311-9f3e5ea76413 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:52.940687 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 368/724] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:12:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:52.946305 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ecf4e-eb62-4ca3-8311-9f3e5ea76413 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:52.984889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:53.023451 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8dd22-244e-4180-a670-95857ebdb6cd None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:53.063742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1c909-5ddb-426d-b1c1-e0a164a96be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:53.064382 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1c909-5ddb-426d-b1c1-e0a164a96be3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:53.064642 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1c909-5ddb-426d-b1c1-e0a164a96be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:53.064904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1c909-5ddb-426d-b1c1-e0a164a96be3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:53.065665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1c909-5ddb-426d-b1c1-e0a164a96be3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:53.067702 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1c909-5ddb-426d-b1c1-e0a164a96be3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:53.073408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ecf4e-eb62-4ca3-8311-9f3e5ea76413 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-ded1c909-5ddb-426d-b1c1-e0a164a96be3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:53.078479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:53.079510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:53.080338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 357/725] 10.222.0.18 () {56 vars in 1105 bytes} [Fri Oct 14 02:12:52 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:53.087756 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1c909-5ddb-426d-b1c1-e0a164a96be3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:53.105753 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:53.127752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469b561-58e1-4385-9128-51ad07fe0268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:53.128575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469b561-58e1-4385-9128-51ad07fe0268 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:53.128851 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469b561-58e1-4385-9128-51ad07fe0268 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:53.129123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469b561-58e1-4385-9128-51ad07fe0268 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:53.129751 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469b561-58e1-4385-9128-51ad07fe0268 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:53.130091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469b561-58e1-4385-9128-51ad07fe0268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:53.131498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469b561-58e1-4385-9128-51ad07fe0268 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:53.141419 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 369/726] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Oct 14 02:12:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:53.142765 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469b561-58e1-4385-9128-51ad07fe0268 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:53.222954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1c909-5ddb-426d-b1c1-e0a164a96be3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:53.245762 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:53.251146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469b561-58e1-4385-9128-51ad07fe0268 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:53.269949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e00c4-ed18-4766-823a-57f720344969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:53.271009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e00c4-ed18-4766-823a-57f720344969 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:53.271485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e00c4-ed18-4766-823a-57f720344969 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:53.271987 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e00c4-ed18-4766-823a-57f720344969 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:53.272819 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e00c4-ed18-4766-823a-57f720344969 None admin] RBAC: Adding query filter params (name=image) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:53.273308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e00c4-ed18-4766-823a-57f720344969 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:53.274879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e00c4-ed18-4766-823a-57f720344969 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:53.286926 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 358/727] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:53.289066 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e00c4-ed18-4766-823a-57f720344969 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:53.292974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1aabcd-714d-4ab7-ae74-a4df094b5bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:53.294317 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aabcd-714d-4ab7-ae74-a4df094b5bdd None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:53.294317 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aabcd-714d-4ab7-ae74-a4df094b5bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:53.294317 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aabcd-714d-4ab7-ae74-a4df094b5bdd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:53.295094 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1aabcd-714d-4ab7-ae74-a4df094b5bdd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:53.296544 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1aabcd-714d-4ab7-ae74-a4df094b5bdd None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:53.327595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 370/728] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:12:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:12:53.366715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee9c3e-05e0-42a9-a8ae-44d8dada13da None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:53.367054 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee9c3e-05e0-42a9-a8ae-44d8dada13da None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:53.367376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee9c3e-05e0-42a9-a8ae-44d8dada13da None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:53.370080 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 371/729] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:12:53.376187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61d95a-131b-45ac-bd74-e88d9e45c130 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:53.376567 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61d95a-131b-45ac-bd74-e88d9e45c130 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:53.376739 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61d95a-131b-45ac-bd74-e88d9e45c130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:53.392390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:53.408831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fb61d95a-131b-45ac-bd74-e88d9e45c130 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:53.410951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fb61d95a-131b-45ac-bd74-e88d9e45c130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:53.422335 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e00c4-ed18-4766-823a-57f720344969 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:53.472160 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c367e-0ea4-45da-9ba0-c5d424f7e5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:53.473138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c367e-0ea4-45da-9ba0-c5d424f7e5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:53.473453 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c367e-0ea4-45da-9ba0-c5d424f7e5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:53.473821 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c367e-0ea4-45da-9ba0-c5d424f7e5b7 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:53.474569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c367e-0ea4-45da-9ba0-c5d424f7e5b7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:53.475093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c367e-0ea4-45da-9ba0-c5d424f7e5b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:53.476557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c367e-0ea4-45da-9ba0-c5d424f7e5b7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:53.488688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 359/730] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:53 2022] GET /identity/v3/services?type=image => generated 122 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:53.509560 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 372/731] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:12:53.786050 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c367e-0ea4-45da-9ba0-c5d424f7e5b7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:53.892989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c367e-0ea4-45da-9ba0-c5d424f7e5b7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:53.928713 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688f258-f353-4f7e-91e2-a3c024a091a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:53.929729 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688f258-f353-4f7e-91e2-a3c024a091a7 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:53.930203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688f258-f353-4f7e-91e2-a3c024a091a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:53.930666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688f258-f353-4f7e-91e2-a3c024a091a7 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:53.931800 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688f258-f353-4f7e-91e2-a3c024a091a7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:53.933446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688f258-f353-4f7e-91e2-a3c024a091a7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:53.938600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:53.952748 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 360/732] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Oct 14 02:12:53 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:54.860301 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65232ab3-022f-4188-9462-408c11f3b2e7 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:54.860819 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65232ab3-022f-4188-9462-408c11f3b2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:54.860819 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65232ab3-022f-4188-9462-408c11f3b2e7 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:54.862526 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 373/733] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:54.868239 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c3faf-aaa3-44bf-abb8-9dbf6b56c634 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:54.868706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c3faf-aaa3-44bf-abb8-9dbf6b56c634 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:54.868780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c3faf-aaa3-44bf-abb8-9dbf6b56c634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:54.887799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-648c3faf-aaa3-44bf-abb8-9dbf6b56c634 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:54.889211 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-648c3faf-aaa3-44bf-abb8-9dbf6b56c634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:54.964310 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 361/734] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:55.239013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65232ab3-022f-4188-9462-408c11f3b2e7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:55.329656 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:55.349474 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65232ab3-022f-4188-9462-408c11f3b2e7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:55.387021 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6448f-69e1-49e3-8065-91c503c9feff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:55.388029 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6448f-69e1-49e3-8065-91c503c9feff None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:55.388533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6448f-69e1-49e3-8065-91c503c9feff None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:55.389007 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6448f-69e1-49e3-8065-91c503c9feff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:55.389803 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6448f-69e1-49e3-8065-91c503c9feff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:55.391416 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6448f-69e1-49e3-8065-91c503c9feff None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-30d6448f-69e1-49e3-8065-91c503c9feff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:55.400509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:55.401436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:55.402240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:55.402240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:55.402240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:55.402240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:55.402240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:55.402240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:55.402240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:55.402240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:55.402240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:55.402240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:55.402240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:55.402240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:55.402240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:55.402240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:55.405602 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 374/735] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Oct 14 02:12:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:55.411367 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c3faf-aaa3-44bf-abb8-9dbf6b56c634 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:55.520750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c3faf-aaa3-44bf-abb8-9dbf6b56c634 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:55.548705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:55.556995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd90b5-0f87-47b2-ab68-488012c802ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:55.557883 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd90b5-0f87-47b2-ab68-488012c802ba None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:55.558379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd90b5-0f87-47b2-ab68-488012c802ba None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:55.558914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd90b5-0f87-47b2-ab68-488012c802ba None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:55.559816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd90b5-0f87-47b2-ab68-488012c802ba None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:55.560866 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd90b5-0f87-47b2-ab68-488012c802ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:55.562816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd90b5-0f87-47b2-ab68-488012c802ba None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:55.568429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 362/736] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Oct 14 02:12:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:55.574452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6448f-69e1-49e3-8065-91c503c9feff None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:55.576468 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0dd0-2bfc-4456-9323-7d8f36ccba1e None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:55.576893 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0dd0-2bfc-4456-9323-7d8f36ccba1e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:55.577297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0dd0-2bfc-4456-9323-7d8f36ccba1e None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:55.579038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 363/737] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:55.584739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fb92e-4a5d-4546-992c-b6287d102580 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:55.585259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fb92e-4a5d-4546-992c-b6287d102580 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:55.585676 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fb92e-4a5d-4546-992c-b6287d102580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:55.606414 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-041fb92e-4a5d-4546-992c-b6287d102580 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:55.608210 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-041fb92e-4a5d-4546-992c-b6287d102580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:55.695111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 364/738] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:12:55.699017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6448f-69e1-49e3-8065-91c503c9feff None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:55.719894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118edf-221a-421d-b2e1-589d04d8df38 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:55.720196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118edf-221a-421d-b2e1-589d04d8df38 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:55.720436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118edf-221a-421d-b2e1-589d04d8df38 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:55.722029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 365/739] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:12:55.732498 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390343c-5e9c-4fde-99a7-e2df0293d22e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:55.732788 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390343c-5e9c-4fde-99a7-e2df0293d22e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:55.733161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390343c-5e9c-4fde-99a7-e2df0293d22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:55.734959 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa360b-f6ee-4e58-86d1-637a4fdfcb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:55.736010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa360b-f6ee-4e58-86d1-637a4fdfcb80 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:55.736440 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa360b-f6ee-4e58-86d1-637a4fdfcb80 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:55.736721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa360b-f6ee-4e58-86d1-637a4fdfcb80 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:55.737411 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa360b-f6ee-4e58-86d1-637a4fdfcb80 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:55.737721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa360b-f6ee-4e58-86d1-637a4fdfcb80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:55.739101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa360b-f6ee-4e58-86d1-637a4fdfcb80 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:55.744999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 375/740] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Oct 14 02:12:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:55.751101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa360b-f6ee-4e58-86d1-637a4fdfcb80 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:55.760685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-8390343c-5e9c-4fde-99a7-e2df0293d22e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:55.761870 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8390343c-5e9c-4fde-99a7-e2df0293d22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:55.768695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:55.843765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 366/741] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:55.866749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa360b-f6ee-4e58-86d1-637a4fdfcb80 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:55.902214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223bfa6-240b-47bd-954d-d1667456f711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:55.903195 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223bfa6-240b-47bd-954d-d1667456f711 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:55.903680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223bfa6-240b-47bd-954d-d1667456f711 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:55.904234 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223bfa6-240b-47bd-954d-d1667456f711 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:55.905154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223bfa6-240b-47bd-954d-d1667456f711 None admin] RBAC: Adding query filter params (interface=admin, service_id=c547dc458810462a865b9d929a05e795, region_id=RegionOne) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:55.905663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223bfa6-240b-47bd-954d-d1667456f711 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:55.907261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223bfa6-240b-47bd-954d-d1667456f711 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:55.913027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 376/742] 10.222.0.18 () {56 vars in 1249 bytes} [Fri Oct 14 02:12:55 2022] GET /identity/v3/endpoints?service_id=c547dc458810462a865b9d929a05e795&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:12:55.918924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390343c-5e9c-4fde-99a7-e2df0293d22e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:55.968327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:56.004777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223bfa6-240b-47bd-954d-d1667456f711 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:56.030908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390343c-5e9c-4fde-99a7-e2df0293d22e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:56.073916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225a65a-cafc-47f3-8c4c-0ee4cd9cad59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:56.075249 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225a65a-cafc-47f3-8c4c-0ee4cd9cad59 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:56.076838 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225a65a-cafc-47f3-8c4c-0ee4cd9cad59 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:56.077671 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225a65a-cafc-47f3-8c4c-0ee4cd9cad59 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:56.078568 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6225a65a-cafc-47f3-8c4c-0ee4cd9cad59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:56.080447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6225a65a-cafc-47f3-8c4c-0ee4cd9cad59 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:56.089039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 367/743] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Oct 14 02:12:55 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:12:56.104643 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:56.131318 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225a65a-cafc-47f3-8c4c-0ee4cd9cad59 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:56.141503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223bfa6-240b-47bd-954d-d1667456f711 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:56.193787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8709a20-619f-478d-9c1f-f20978ca103e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:56.194699 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8709a20-619f-478d-9c1f-f20978ca103e None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:56.194979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8709a20-619f-478d-9c1f-f20978ca103e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:56.195279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8709a20-619f-478d-9c1f-f20978ca103e None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:56.196111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8709a20-619f-478d-9c1f-f20978ca103e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:56.197983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8709a20-619f-478d-9c1f-f20978ca103e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:56.200735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: WARNING keystone.server.flask.common [None req-e8709a20-619f-478d-9c1f-f20978ca103e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 14 02:12:56.220297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.sql.core [None req-e8709a20-619f-478d-9c1f-f20978ca103e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 14 02:12:56.220297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 02:12:56.220297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [parameters: {'id': '153e16b07fea431e8d863c20c2f654e9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 14 02:12:56.220297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81361) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 02:12:56.237482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-e8709a20-619f-478d-9c1f-f20978ca103e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 153e16b07fea431e8d863c20c2f654e9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 153e16b07fea431e8d863c20c2f654e9. Oct 14 02:12:56.237482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:56.237482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:12:56.237482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:12:56.237482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:12:56.237482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:12:56.237482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:12:56.237482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:12:56.237482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:12:56.237482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:12:56.237482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:12:56.237482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:12:56.237482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application result.read() Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 14 02:12:56.238409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application yield resource Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise self.value Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application session.commit() Oct 14 02:12:56.239445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.session.flush() Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application compat.raise_( Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 02:12:56.240356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 02:12:56.241243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application result.read() Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:12:56.242071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [parameters: {'id': '153e16b07fea431e8d863c20c2f654e9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:56.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:56.245939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:56.248904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:56.248904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 14 02:12:56.248904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 14 02:12:56.248904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:56.248904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:56.248904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 14 02:12:56.248904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 14 02:12:56.248904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 153e16b07fea431e8d863c20c2f654e9. Oct 14 02:12:56.248904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:56.248904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 377/744] 10.222.0.18 () {60 vars in 1140 bytes} [Fri Oct 14 02:12:56 2022] POST /identity/v3/projects => generated 262 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:56.248904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340dbb2-1f39-4cc2-9a14-43f37df59d3f None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:56.248904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340dbb2-1f39-4cc2-9a14-43f37df59d3f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:56.251246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340dbb2-1f39-4cc2-9a14-43f37df59d3f None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:56.251246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 378/745] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:56.251246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340dbb2-1f39-4cc2-9a14-43f37df59d3f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:56.269094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225a65a-cafc-47f3-8c4c-0ee4cd9cad59 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:56.281930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:56.310175 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2a0bd-0a26-4ee2-9735-899bb39ca0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:56.311392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2a0bd-0a26-4ee2-9735-899bb39ca0b0 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:56.311840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2a0bd-0a26-4ee2-9735-899bb39ca0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:56.312109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2a0bd-0a26-4ee2-9735-899bb39ca0b0 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:56.313151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2a0bd-0a26-4ee2-9735-899bb39ca0b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:56.314505 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2a0bd-0a26-4ee2-9735-899bb39ca0b0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:56.330488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 368/746] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Oct 14 02:12:56 2022] POST /identity/v3/services => generated 242 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:12:56.339848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e2071-70f9-44bf-837d-a7b5eb8269a9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:56.340129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e2071-70f9-44bf-837d-a7b5eb8269a9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:56.340401 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e2071-70f9-44bf-837d-a7b5eb8269a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:56.370056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-825e2071-70f9-44bf-837d-a7b5eb8269a9 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:56.371902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-825e2071-70f9-44bf-837d-a7b5eb8269a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:56.393217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340dbb2-1f39-4cc2-9a14-43f37df59d3f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:56.425408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:56.437636 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9688e-de5e-4d5b-bdde-6331a9646944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:56.438305 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9688e-de5e-4d5b-bdde-6331a9646944 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:56.438684 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9688e-de5e-4d5b-bdde-6331a9646944 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:56.440015 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9688e-de5e-4d5b-bdde-6331a9646944 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:56.448137 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9688e-de5e-4d5b-bdde-6331a9646944 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:56.449855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9688e-de5e-4d5b-bdde-6331a9646944 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:56.456977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 369/747] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:56 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-80c9688e-de5e-4d5b-bdde-6331a9646944 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:12:56.469503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:56.470473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:56.471371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:12:56.471371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:12:56.471371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:12:56.471371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:12:56.471371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:12:56.471371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:12:56.471371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:12:56.471371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:56.473989 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 379/748] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:56.480360 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e2071-70f9-44bf-837d-a7b5eb8269a9 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:56.589553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:56.593047 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e2071-70f9-44bf-837d-a7b5eb8269a9 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:56.629750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186eeac-f410-40e7-83f1-8556608c9e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:56.630288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186eeac-f410-40e7-83f1-8556608c9e92 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:56.630630 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186eeac-f410-40e7-83f1-8556608c9e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:56.630911 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186eeac-f410-40e7-83f1-8556608c9e92 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:56.631617 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186eeac-f410-40e7-83f1-8556608c9e92 None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:56.631922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186eeac-f410-40e7-83f1-8556608c9e92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:56.633433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186eeac-f410-40e7-83f1-8556608c9e92 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:56.651691 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 370/749] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Oct 14 02:12:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:56.757016 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9688e-de5e-4d5b-bdde-6331a9646944 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:56.871956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9688e-de5e-4d5b-bdde-6331a9646944 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:56.908481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d4e5c-54d3-4839-97a0-49faaca45692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:56.909199 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d4e5c-54d3-4839-97a0-49faaca45692 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:56.909582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d4e5c-54d3-4839-97a0-49faaca45692 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:56.910036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d4e5c-54d3-4839-97a0-49faaca45692 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:56.910716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d4e5c-54d3-4839-97a0-49faaca45692 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:56.912293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d4e5c-54d3-4839-97a0-49faaca45692 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a22d4e5c-54d3-4839-97a0-49faaca45692 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:56.921992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:56.923239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:56.924240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:56.924240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:56.924240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:56.924240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:56.924240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:56.924240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:56.924240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:56.924240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:56.924240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:56.924240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:56.924240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:56.924240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:56.924240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:12:56.924240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:56.925105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 380/750] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:12:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:56.930615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186eeac-f410-40e7-83f1-8556608c9e92 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:57.042539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186eeac-f410-40e7-83f1-8556608c9e92 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:57.077964 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb364989-aea1-4741-bcc8-4cc5edfb420a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:57.078980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb364989-aea1-4741-bcc8-4cc5edfb420a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:57.079508 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb364989-aea1-4741-bcc8-4cc5edfb420a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:57.079979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb364989-aea1-4741-bcc8-4cc5edfb420a None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:57.080807 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb364989-aea1-4741-bcc8-4cc5edfb420a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:57.081306 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb364989-aea1-4741-bcc8-4cc5edfb420a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:57.082885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb364989-aea1-4741-bcc8-4cc5edfb420a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:57.088491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 371/751] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Oct 14 02:12:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:57.096255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d4e5c-54d3-4839-97a0-49faaca45692 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:57.152560 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:57.201855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d4e5c-54d3-4839-97a0-49faaca45692 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:57.237004 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285db3ce-df37-495e-a240-03e193caf4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:57.237838 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285db3ce-df37-495e-a240-03e193caf4ea None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:57.238214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285db3ce-df37-495e-a240-03e193caf4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:57.238583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285db3ce-df37-495e-a240-03e193caf4ea None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:57.239346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285db3ce-df37-495e-a240-03e193caf4ea None admin] RBAC: Adding query filter params (type=compute) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:57.239749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285db3ce-df37-495e-a240-03e193caf4ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:57.241235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285db3ce-df37-495e-a240-03e193caf4ea None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:57.247049 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 381/752] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Oct 14 02:12:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:57.256163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb364989-aea1-4741-bcc8-4cc5edfb420a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:57.283178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:57.368829 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb364989-aea1-4741-bcc8-4cc5edfb420a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:57.405980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a982c-64fa-4816-ae62-0fd453adb576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:57.406915 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a982c-64fa-4816-ae62-0fd453adb576 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:57.407401 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a982c-64fa-4816-ae62-0fd453adb576 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:57.407823 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a982c-64fa-4816-ae62-0fd453adb576 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:57.408670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a982c-64fa-4816-ae62-0fd453adb576 None admin] RBAC: Adding query filter params (interface=public, service_id=cbc59f9f83f6465887cc3282af4f96bb, region_id=RegionOne) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:57.409126 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a982c-64fa-4816-ae62-0fd453adb576 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:57.410654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a982c-64fa-4816-ae62-0fd453adb576 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:57.416747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 372/753] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Oct 14 02:12:57 2022] GET /identity/v3/endpoints?service_id=cbc59f9f83f6465887cc3282af4f96bb&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:57.423372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285db3ce-df37-495e-a240-03e193caf4ea None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:57.526907 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285db3ce-df37-495e-a240-03e193caf4ea None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:57.543459 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:57.562842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01e464-7198-4aba-ac7c-8946b30338f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:57.563495 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01e464-7198-4aba-ac7c-8946b30338f6 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:57.563775 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01e464-7198-4aba-ac7c-8946b30338f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:57.564048 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01e464-7198-4aba-ac7c-8946b30338f6 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:57.564934 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01e464-7198-4aba-ac7c-8946b30338f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:57.566312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01e464-7198-4aba-ac7c-8946b30338f6 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:57.573416 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 382/754] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Oct 14 02:12:57 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:12:58.291731 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cdeb7-eef0-4f8d-b4f1-0c3fb18dd8cf None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:58.292276 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cdeb7-eef0-4f8d-b4f1-0c3fb18dd8cf None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:58.292276 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cdeb7-eef0-4f8d-b4f1-0c3fb18dd8cf None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:58.293847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 373/755] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:58.299899 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50043f14-3f2a-4b7e-ba49-1d0a1f0eb212 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:58.300326 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50043f14-3f2a-4b7e-ba49-1d0a1f0eb212 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:58.300489 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50043f14-3f2a-4b7e-ba49-1d0a1f0eb212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:58.318755 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-50043f14-3f2a-4b7e-ba49-1d0a1f0eb212 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:58.320296 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-50043f14-3f2a-4b7e-ba49-1d0a1f0eb212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:58.389542 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 383/756] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:58 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:58.638338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94ea2d-b5f0-4da7-8296-011765223504 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:58.638706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94ea2d-b5f0-4da7-8296-011765223504 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:58.638887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94ea2d-b5f0-4da7-8296-011765223504 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:58.640424 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 374/757] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:58.645747 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021ec84-f40d-412b-acbc-2ca338a202d5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:58.646031 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021ec84-f40d-412b-acbc-2ca338a202d5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:58.646307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021ec84-f40d-412b-acbc-2ca338a202d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:58.660183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94ea2d-b5f0-4da7-8296-011765223504 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:58.665060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2021ec84-f40d-412b-acbc-2ca338a202d5 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:58.666224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2021ec84-f40d-412b-acbc-2ca338a202d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:58.749252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 384/758] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:58 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:58.761311 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:58.775951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94ea2d-b5f0-4da7-8296-011765223504 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:58.816423 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a08a47-072c-4cd0-bf25-0ac7b1f30c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:58.817062 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a08a47-072c-4cd0-bf25-0ac7b1f30c4f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:58.817394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a08a47-072c-4cd0-bf25-0ac7b1f30c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:58.817669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a08a47-072c-4cd0-bf25-0ac7b1f30c4f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:58.818248 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a08a47-072c-4cd0-bf25-0ac7b1f30c4f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:58.819708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a08a47-072c-4cd0-bf25-0ac7b1f30c4f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-04a08a47-072c-4cd0-bf25-0ac7b1f30c4f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:58.829441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:58.830429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:12:58.831266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 375/759] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Oct 14 02:12:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:58.836879 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021ec84-f40d-412b-acbc-2ca338a202d5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:58.879484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:58.938352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021ec84-f40d-412b-acbc-2ca338a202d5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:58.972716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11346673-47f2-4dd5-8b4f-05397e5a04c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:58.973320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11346673-47f2-4dd5-8b4f-05397e5a04c6 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:58.973594 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11346673-47f2-4dd5-8b4f-05397e5a04c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:58.973867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11346673-47f2-4dd5-8b4f-05397e5a04c6 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:58.974498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11346673-47f2-4dd5-8b4f-05397e5a04c6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:58.974795 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11346673-47f2-4dd5-8b4f-05397e5a04c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:58.976289 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11346673-47f2-4dd5-8b4f-05397e5a04c6 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:58.982795 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 385/760] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Oct 14 02:12:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:58.989305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a08a47-072c-4cd0-bf25-0ac7b1f30c4f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:59.028962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11346673-47f2-4dd5-8b4f-05397e5a04c6 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:59.100321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a08a47-072c-4cd0-bf25-0ac7b1f30c4f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:59.132110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11346673-47f2-4dd5-8b4f-05397e5a04c6 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:59.134518 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:59.135492 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a36a48-f71b-4fea-87b9-278340727dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:59.136102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a36a48-f71b-4fea-87b9-278340727dbe None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:59.136392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a36a48-f71b-4fea-87b9-278340727dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:59.136690 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a36a48-f71b-4fea-87b9-278340727dbe None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:59.137283 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a36a48-f71b-4fea-87b9-278340727dbe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:59.137581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a36a48-f71b-4fea-87b9-278340727dbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:59.138970 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a36a48-f71b-4fea-87b9-278340727dbe None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:59.143965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 376/761] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Oct 14 02:12:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:59.149558 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71e252-d224-4d7a-a40e-f118bf7709ad None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:59.149839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71e252-d224-4d7a-a40e-f118bf7709ad None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:59.150106 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71e252-d224-4d7a-a40e-f118bf7709ad None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:59.151629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 377/762] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:59.152314 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71e252-d224-4d7a-a40e-f118bf7709ad None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:59.171597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:59.172610 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb23c3-e006-4ed0-9e6f-be9c0d4ff2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:59.173249 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb23c3-e006-4ed0-9e6f-be9c0d4ff2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:59.173544 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb23c3-e006-4ed0-9e6f-be9c0d4ff2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:59.173806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb23c3-e006-4ed0-9e6f-be9c0d4ff2c5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:59.174411 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb23c3-e006-4ed0-9e6f-be9c0d4ff2c5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:59.175770 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb23c3-e006-4ed0-9e6f-be9c0d4ff2c5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-c5bb23c3-e006-4ed0-9e6f-be9c0d4ff2c5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:59.184977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:59.186551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:59.187857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 386/763] 10.222.0.18 () {56 vars in 1105 bytes} [Fri Oct 14 02:12:59 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:59.188705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a8a87-f956-47e6-be73-ec60b195cb8c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:59.188705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a8a87-f956-47e6-be73-ec60b195cb8c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:59.188705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a8a87-f956-47e6-be73-ec60b195cb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:59.207395 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-861a8a87-f956-47e6-be73-ec60b195cb8c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:12:59.208575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-861a8a87-f956-47e6-be73-ec60b195cb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:12:59.265546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71e252-d224-4d7a-a40e-f118bf7709ad None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:59.287101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 387/764] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:59 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:12:59.287101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a8a87-f956-47e6-be73-ec60b195cb8c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:59.305854 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c07069-0a00-448f-83ab-0d161c0b41f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:59.306468 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c07069-0a00-448f-83ab-0d161c0b41f8 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:59.306739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c07069-0a00-448f-83ab-0d161c0b41f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:59.307385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c07069-0a00-448f-83ab-0d161c0b41f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:59.308048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c07069-0a00-448f-83ab-0d161c0b41f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:59.309428 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c07069-0a00-448f-83ab-0d161c0b41f8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:59.326630 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 378/765] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:12:59 2022] POST /identity/v3/endpoints => generated 325 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:12:59.410794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:59.419421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a8a87-f956-47e6-be73-ec60b195cb8c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:59.467487 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ece3a8-523f-4e96-8452-98f2cf213e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:59.468457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece3a8-523f-4e96-8452-98f2cf213e46 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:59.468931 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece3a8-523f-4e96-8452-98f2cf213e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:59.469411 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece3a8-523f-4e96-8452-98f2cf213e46 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:59.470262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece3a8-523f-4e96-8452-98f2cf213e46 None admin] RBAC: Adding query filter params (name=image) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:59.470756 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece3a8-523f-4e96-8452-98f2cf213e46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:59.472360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece3a8-523f-4e96-8452-98f2cf213e46 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:59.478724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 388/766] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:59.485192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c07069-0a00-448f-83ab-0d161c0b41f8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:59.569762 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:59.576472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ece3a8-523f-4e96-8452-98f2cf213e46 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:59.600155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c07069-0a00-448f-83ab-0d161c0b41f8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:59.641081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2612a-d842-42ca-a300-8d985e89e92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:59.641800 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2612a-d842-42ca-a300-8d985e89e92a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:59.642085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2612a-d842-42ca-a300-8d985e89e92a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:59.642360 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2612a-d842-42ca-a300-8d985e89e92a None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:59.642994 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2612a-d842-42ca-a300-8d985e89e92a None admin] RBAC: Adding query filter params (type=image) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:59.643326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2612a-d842-42ca-a300-8d985e89e92a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:59.645364 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2612a-d842-42ca-a300-8d985e89e92a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:59.655156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 379/767] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:12:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:59.658868 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2612a-d842-42ca-a300-8d985e89e92a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:59.697463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ece3a8-523f-4e96-8452-98f2cf213e46 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:59.746141 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a61842-0e49-4b88-abd4-dccc0517a60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:59.747136 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a61842-0e49-4b88-abd4-dccc0517a60f None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:59.747554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a61842-0e49-4b88-abd4-dccc0517a60f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:59.747926 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a61842-0e49-4b88-abd4-dccc0517a60f None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:59.748972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a61842-0e49-4b88-abd4-dccc0517a60f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:59.751427 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a61842-0e49-4b88-abd4-dccc0517a60f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:59.770889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.sql.core [None req-18a61842-0e49-4b88-abd4-dccc0517a60f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 14 02:12:59.770889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 14 02:12:59.770889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [parameters: {'user_id': 'e7eb458522a447419194dcf915f3c1a9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 14 02:12:59.770889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81361) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-18a61842-0e49-4b88-abd4-dccc0517a60f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application result.read() Oct 14 02:12:59.781911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:12:59.782817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application yield resource Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise self.value Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application session.commit() Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 02:12:59.783962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.session.flush() Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application compat.raise_( Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 02:12:59.784863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:12:59.785720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application result.read() Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:12:59.786523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e7eb458522a447419194dcf915f3c1a9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:59.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 14 02:12:59.795974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 14 02:12:59.798149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 14 02:12:59.798149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 14 02:12:59.798149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 02:12:59.798149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 02:12:59.798149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 14 02:12:59.798149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:59.798149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 389/768] 10.222.0.18 () {60 vars in 1131 bytes} [Fri Oct 14 02:12:59 2022] POST /identity/v3/users => generated 160 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:12:59.799012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2612a-d842-42ca-a300-8d985e89e92a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:59.799195 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a61842-0e49-4b88-abd4-dccc0517a60f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:59.821921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:59.834199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f350849-b8a0-4016-9621-3a88078b9cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:59.834843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f350849-b8a0-4016-9621-3a88078b9cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:59.835155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f350849-b8a0-4016-9621-3a88078b9cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:59.835429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f350849-b8a0-4016-9621-3a88078b9cd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:59.836130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f350849-b8a0-4016-9621-3a88078b9cd5 None admin] RBAC: Adding query filter params (interface=public, service_id=00eea3d26e20461fbd1513bae9f2b1fc, region_id=RegionOne) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:12:59.836442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f350849-b8a0-4016-9621-3a88078b9cd5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:59.838000 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f350849-b8a0-4016-9621-3a88078b9cd5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:59.845376 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 380/769] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Oct 14 02:12:59 2022] GET /identity/v3/endpoints?service_id=00eea3d26e20461fbd1513bae9f2b1fc&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:12:59.846491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaac9ab-fd66-44f1-bafa-54aba52ec9c4 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:59.846780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaac9ab-fd66-44f1-bafa-54aba52ec9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:59.847053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaac9ab-fd66-44f1-bafa-54aba52ec9c4 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:59.848398 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 381/770] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:12:59.853380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaac9ab-fd66-44f1-bafa-54aba52ec9c4 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:12:59.858535 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:12:59.916347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a61842-0e49-4b88-abd4-dccc0517a60f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:59.953088 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f049bcd-71f8-4959-a07a-bbd6528d2584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:12:59.953729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f049bcd-71f8-4959-a07a-bbd6528d2584 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:59.954141 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f049bcd-71f8-4959-a07a-bbd6528d2584 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:59.954336 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f049bcd-71f8-4959-a07a-bbd6528d2584 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:12:59.962264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f049bcd-71f8-4959-a07a-bbd6528d2584 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:12:59.963279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaac9ab-fd66-44f1-bafa-54aba52ec9c4 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:12:59.963975 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f049bcd-71f8-4959-a07a-bbd6528d2584 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-5f049bcd-71f8-4959-a07a-bbd6528d2584 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:12:59.978767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:12:59.984870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:12:59.986026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:12:59.986026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:12:59.986026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:12:59.986026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:12:59.986026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:12:59.986026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:12:59.986026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:12:59.986026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:12:59.986026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 02:12:59.986026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:12:59.986026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 390/771] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:12:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:12:59.986026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025605d2-c3b1-48a2-97bf-5655d9878220 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:12:59.986026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025605d2-c3b1-48a2-97bf-5655d9878220 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:12:59.986877 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025605d2-c3b1-48a2-97bf-5655d9878220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:00.001219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-025605d2-c3b1-48a2-97bf-5655d9878220 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:00.002730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-025605d2-c3b1-48a2-97bf-5655d9878220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:00.005985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47608247-9766-4b72-9ec1-62a70b3712ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:00.006710 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47608247-9766-4b72-9ec1-62a70b3712ed None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:00.007137 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47608247-9766-4b72-9ec1-62a70b3712ed None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:00.007509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47608247-9766-4b72-9ec1-62a70b3712ed None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:00.008213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47608247-9766-4b72-9ec1-62a70b3712ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:00.009669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47608247-9766-4b72-9ec1-62a70b3712ed None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:00.016773 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 382/772] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Oct 14 02:12:59 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:13:00.017660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47608247-9766-4b72-9ec1-62a70b3712ed None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:00.095920 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:00.116047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 391/773] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:12:59 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:00.139980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47608247-9766-4b72-9ec1-62a70b3712ed None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:00.182723 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f212cb-5116-4003-bfeb-e6bb7a2387f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:00.183573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f212cb-5116-4003-bfeb-e6bb7a2387f3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:00.183959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f212cb-5116-4003-bfeb-e6bb7a2387f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:00.184345 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f212cb-5116-4003-bfeb-e6bb7a2387f3 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:00.185840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f212cb-5116-4003-bfeb-e6bb7a2387f3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:00.186338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f212cb-5116-4003-bfeb-e6bb7a2387f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:00.187952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f212cb-5116-4003-bfeb-e6bb7a2387f3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:00.229321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 383/774] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Oct 14 02:13:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:00.400027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025605d2-c3b1-48a2-97bf-5655d9878220 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:00.412332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:00.509919 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025605d2-c3b1-48a2-97bf-5655d9878220 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:00.661401 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edcfa0-a605-4e09-b03c-d7eb3c4d2a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:00.662039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edcfa0-a605-4e09-b03c-d7eb3c4d2a95 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:00.662312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edcfa0-a605-4e09-b03c-d7eb3c4d2a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:00.662586 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edcfa0-a605-4e09-b03c-d7eb3c4d2a95 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:00.663191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edcfa0-a605-4e09-b03c-d7eb3c4d2a95 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:00.664567 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edcfa0-a605-4e09-b03c-d7eb3c4d2a95 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-29edcfa0-a605-4e09-b03c-d7eb3c4d2a95 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:00.673523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:00.674465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:00.675735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:00.675735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:00.675735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:00.675735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:00.675735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:00.675735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:00.675735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:00.675735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:00.675735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:00.675735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:00.675735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:13:00.675735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:13:00.675735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:13:00.675735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:13:00.679138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 392/775] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:13:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:00.680603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f212cb-5116-4003-bfeb-e6bb7a2387f3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:00.789034 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f212cb-5116-4003-bfeb-e6bb7a2387f3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:00.816261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:00.826156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720f311-e4df-495d-a0f1-c12dc0d7c0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:00.826910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720f311-e4df-495d-a0f1-c12dc0d7c0a3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:00.827235 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720f311-e4df-495d-a0f1-c12dc0d7c0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:00.827513 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720f311-e4df-495d-a0f1-c12dc0d7c0a3 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:00.828150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720f311-e4df-495d-a0f1-c12dc0d7c0a3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:00.828454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720f311-e4df-495d-a0f1-c12dc0d7c0a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:00.829906 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720f311-e4df-495d-a0f1-c12dc0d7c0a3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:00.839408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 384/776] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Oct 14 02:13:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:00.840905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edcfa0-a605-4e09-b03c-d7eb3c4d2a95 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:00.916128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:00.942460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edcfa0-a605-4e09-b03c-d7eb3c4d2a95 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:00.978458 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3626d-94e5-4784-9db4-22ad8476ab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:00.979076 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3626d-94e5-4784-9db4-22ad8476ab95 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:00.979362 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3626d-94e5-4784-9db4-22ad8476ab95 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:00.979632 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3626d-94e5-4784-9db4-22ad8476ab95 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:00.980260 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3626d-94e5-4784-9db4-22ad8476ab95 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:00.980559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3626d-94e5-4784-9db4-22ad8476ab95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:00.982459 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3626d-94e5-4784-9db4-22ad8476ab95 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:00.990085 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 393/777] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Oct 14 02:13:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:00.995753 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720f311-e4df-495d-a0f1-c12dc0d7c0a3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:01.094036 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720f311-e4df-495d-a0f1-c12dc0d7c0a3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:01.127519 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f535e-5844-4109-8c25-ce07d995e0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:01.128132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f535e-5844-4109-8c25-ce07d995e0f8 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:01.128411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f535e-5844-4109-8c25-ce07d995e0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:01.128684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f535e-5844-4109-8c25-ce07d995e0f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:01.129340 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f535e-5844-4109-8c25-ce07d995e0f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:01.130698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f535e-5844-4109-8c25-ce07d995e0f8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:01.151090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 385/778] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:13:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:13:01.583867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33418797-bd97-44c4-8038-9ee1145d85c2 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:01.584388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33418797-bd97-44c4-8038-9ee1145d85c2 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:01.584388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33418797-bd97-44c4-8038-9ee1145d85c2 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:01.585903 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 394/779] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:01.590952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04553bd-eb78-459c-8d3e-6213173a58cb None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:01.591798 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04553bd-eb78-459c-8d3e-6213173a58cb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:01.592071 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04553bd-eb78-459c-8d3e-6213173a58cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:01.611527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d04553bd-eb78-459c-8d3e-6213173a58cb None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:01.612702 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d04553bd-eb78-459c-8d3e-6213173a58cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:01.655984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:01.691111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 386/780] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:01.962143 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33418797-bd97-44c4-8038-9ee1145d85c2 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:02.065595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33418797-bd97-44c4-8038-9ee1145d85c2 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:02.100888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0225e67-a65d-4dc3-b720-4c7977700ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:02.101537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0225e67-a65d-4dc3-b720-4c7977700ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:02.101806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0225e67-a65d-4dc3-b720-4c7977700ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:02.102071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0225e67-a65d-4dc3-b720-4c7977700ba1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:02.103234 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0225e67-a65d-4dc3-b720-4c7977700ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:02.104655 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0225e67-a65d-4dc3-b720-4c7977700ba1 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:02.106442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 395/781] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:13:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:13:02.112658 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04553bd-eb78-459c-8d3e-6213173a58cb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:02.214231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04553bd-eb78-459c-8d3e-6213173a58cb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:02.249203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abd8cf-6abd-42a0-8589-c271f43c246a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:02.249807 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abd8cf-6abd-42a0-8589-c271f43c246a None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:02.250177 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abd8cf-6abd-42a0-8589-c271f43c246a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:02.250443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abd8cf-6abd-42a0-8589-c271f43c246a None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:02.251113 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abd8cf-6abd-42a0-8589-c271f43c246a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:02.251642 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff2974-db53-42f8-ae1c-6096d3800feb None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:02.251923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff2974-db53-42f8-ae1c-6096d3800feb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:02.252188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff2974-db53-42f8-ae1c-6096d3800feb None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:02.252541 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abd8cf-6abd-42a0-8589-c271f43c246a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:02.253644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 396/782] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:13:02.258715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e43ca-44e6-4a5f-8323-11f610b38569 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:02.258993 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e43ca-44e6-4a5f-8323-11f610b38569 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:02.259294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e43ca-44e6-4a5f-8323-11f610b38569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:02.272729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:02.278985 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b49e43ca-44e6-4a5f-8323-11f610b38569 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:02.280394 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b49e43ca-44e6-4a5f-8323-11f610b38569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:02.280777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 387/783] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Oct 14 02:13:02 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:13:02.355480 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 397/784] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:13:02.644045 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abd8cf-6abd-42a0-8589-c271f43c246a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:02.671043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:02.750632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abd8cf-6abd-42a0-8589-c271f43c246a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:02.785953 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7186718-4225-4f72-920a-5b83f53e8e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:02.787103 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7186718-4225-4f72-920a-5b83f53e8e20 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:02.787579 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7186718-4225-4f72-920a-5b83f53e8e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:02.788043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7186718-4225-4f72-920a-5b83f53e8e20 None admin] PATH_INFO: `/v3/services/image` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:02.788809 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7186718-4225-4f72-920a-5b83f53e8e20 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:02.790358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7186718-4225-4f72-920a-5b83f53e8e20 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-e7186718-4225-4f72-920a-5b83f53e8e20 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:02.800674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:02.801686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:02.803842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 388/785] 10.222.0.18 () {56 vars in 1105 bytes} [Fri Oct 14 02:13:02 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:02.808267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e43ca-44e6-4a5f-8323-11f610b38569 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:02.912643 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e43ca-44e6-4a5f-8323-11f610b38569 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:02.946091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59eaf1-4acb-4d89-93e1-a9a539ba4615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:02.946701 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59eaf1-4acb-4d89-93e1-a9a539ba4615 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:02.946980 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59eaf1-4acb-4d89-93e1-a9a539ba4615 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:02.947306 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59eaf1-4acb-4d89-93e1-a9a539ba4615 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:02.947911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59eaf1-4acb-4d89-93e1-a9a539ba4615 None admin] RBAC: Adding query filter params (name=image) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:02.948213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59eaf1-4acb-4d89-93e1-a9a539ba4615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:02.949582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59eaf1-4acb-4d89-93e1-a9a539ba4615 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:02.951462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:02.955774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 398/786] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:13:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:02.961270 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7186718-4225-4f72-920a-5b83f53e8e20 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:03.050049 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:03.064183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7186718-4225-4f72-920a-5b83f53e8e20 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:03.097996 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ebcc7-6a5b-4544-b63e-1716dc66cc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:03.098612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ebcc7-6a5b-4544-b63e-1716dc66cc56 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:03.098893 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ebcc7-6a5b-4544-b63e-1716dc66cc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:03.099204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ebcc7-6a5b-4544-b63e-1716dc66cc56 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:03.099833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ebcc7-6a5b-4544-b63e-1716dc66cc56 None admin] RBAC: Adding query filter params (type=image) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:03.100134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ebcc7-6a5b-4544-b63e-1716dc66cc56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:03.101498 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ebcc7-6a5b-4544-b63e-1716dc66cc56 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:03.110926 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 389/787] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:13:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:03.112483 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59eaf1-4acb-4d89-93e1-a9a539ba4615 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:03.217515 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59eaf1-4acb-4d89-93e1-a9a539ba4615 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:03.253365 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1975728-abd7-478f-847f-5478ab24355c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:03.253976 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1975728-abd7-478f-847f-5478ab24355c None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:03.254253 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1975728-abd7-478f-847f-5478ab24355c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:03.254521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1975728-abd7-478f-847f-5478ab24355c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:03.255223 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1975728-abd7-478f-847f-5478ab24355c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:03.256584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1975728-abd7-478f-847f-5478ab24355c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:03.283103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 399/788] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:13:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:13:03.402517 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812ffe0-2183-478f-8c37-faab9680aa1b None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:03.402882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812ffe0-2183-478f-8c37-faab9680aa1b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:03.403097 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812ffe0-2183-478f-8c37-faab9680aa1b None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:03.405057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 390/789] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:03.410498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56ec34-ac1b-4028-8680-c7aceb98f1b4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:03.410786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56ec34-ac1b-4028-8680-c7aceb98f1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:03.411110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56ec34-ac1b-4028-8680-c7aceb98f1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:03.431309 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-cb56ec34-ac1b-4028-8680-c7aceb98f1b4 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:03.432534 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cb56ec34-ac1b-4028-8680-c7aceb98f1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:03.486604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:03.510329 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 400/790] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:03.775953 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812ffe0-2183-478f-8c37-faab9680aa1b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:03.877970 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812ffe0-2183-478f-8c37-faab9680aa1b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:03.911643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c394a-431d-44fd-ad1f-2c0db1762ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:03.912267 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c394a-431d-44fd-ad1f-2c0db1762ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:03.912539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c394a-431d-44fd-ad1f-2c0db1762ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:03.912804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c394a-431d-44fd-ad1f-2c0db1762ff7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:03.922769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c394a-431d-44fd-ad1f-2c0db1762ff7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-899c394a-431d-44fd-ad1f-2c0db1762ff7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:13:03.927539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:03.928433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:03.929258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:03.929258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:03.929258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:03.929258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:03.929258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:13:03.929258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:13:03.929258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:13:03.929258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:13:03.929258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:13:03.929258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:13:03.929258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:13:03.929258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:03.931135 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 391/791] 10.222.0.18 () {56 vars in 1120 bytes} [Fri Oct 14 02:13:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:03.935558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56ec34-ac1b-4028-8680-c7aceb98f1b4 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:04.037315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56ec34-ac1b-4028-8680-c7aceb98f1b4 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:04.071328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391a32d-bd20-40a9-a3b2-1df9e1eb1f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:04.071943 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391a32d-bd20-40a9-a3b2-1df9e1eb1f4e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:04.072213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391a32d-bd20-40a9-a3b2-1df9e1eb1f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:04.072477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391a32d-bd20-40a9-a3b2-1df9e1eb1f4e None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:04.073134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391a32d-bd20-40a9-a3b2-1df9e1eb1f4e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:04.073428 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391a32d-bd20-40a9-a3b2-1df9e1eb1f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:04.074777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391a32d-bd20-40a9-a3b2-1df9e1eb1f4e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:04.088100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 401/792] 10.222.0.18 () {56 vars in 1115 bytes} [Fri Oct 14 02:13:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:04.089316 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c394a-431d-44fd-ad1f-2c0db1762ff7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:04.105621 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:04.197068 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c394a-431d-44fd-ad1f-2c0db1762ff7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:04.232847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a7e4-28c6-42de-b7b8-4ae38b2aaff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:04.233546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a7e4-28c6-42de-b7b8-4ae38b2aaff4 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:04.233830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a7e4-28c6-42de-b7b8-4ae38b2aaff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:04.234103 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a7e4-28c6-42de-b7b8-4ae38b2aaff4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:04.243967 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087a7e4-28c6-42de-b7b8-4ae38b2aaff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:04.245407 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087a7e4-28c6-42de-b7b8-4ae38b2aaff4 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-9087a7e4-28c6-42de-b7b8-4ae38b2aaff4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:13:04.260260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:04.261150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:13:04.262187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:13:04.262187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:13:04.262187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:04.262187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 392/793] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:13:04.267929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391a32d-bd20-40a9-a3b2-1df9e1eb1f4e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:04.315123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:04.373306 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391a32d-bd20-40a9-a3b2-1df9e1eb1f4e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:04.412074 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a708b5-430d-40b4-82be-678cf1d39e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:04.412687 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a708b5-430d-40b4-82be-678cf1d39e95 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:04.412962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a708b5-430d-40b4-82be-678cf1d39e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:04.413233 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a708b5-430d-40b4-82be-678cf1d39e95 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:04.413884 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a708b5-430d-40b4-82be-678cf1d39e95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:04.414186 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a708b5-430d-40b4-82be-678cf1d39e95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:04.415585 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a708b5-430d-40b4-82be-678cf1d39e95 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:04.428701 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 402/794] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:04.434185 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a7e4-28c6-42de-b7b8-4ae38b2aaff4 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:04.494234 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcedb42-526e-402b-98cc-6952115256fc None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:04.494541 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcedb42-526e-402b-98cc-6952115256fc None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:04.494779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcedb42-526e-402b-98cc-6952115256fc None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:04.496390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 403/795] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:13:04.499384 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:04.501819 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a9f64-c5d8-48b3-8c95-a579a53264ab None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:04.502153 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a9f64-c5d8-48b3-8c95-a579a53264ab None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:04.502362 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a9f64-c5d8-48b3-8c95-a579a53264ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:04.520597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c34a9f64-c5d8-48b3-8c95-a579a53264ab None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:04.521763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c34a9f64-c5d8-48b3-8c95-a579a53264ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:04.544998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a7e4-28c6-42de-b7b8-4ae38b2aaff4 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:04.582495 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540a07f-9551-42e8-b2a7-dc4b793cecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:04.583094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540a07f-9551-42e8-b2a7-dc4b793cecf9 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:04.583348 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540a07f-9551-42e8-b2a7-dc4b793cecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:04.583621 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540a07f-9551-42e8-b2a7-dc4b793cecf9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:04.591474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540a07f-9551-42e8-b2a7-dc4b793cecf9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:04.593088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540a07f-9551-42e8-b2a7-dc4b793cecf9 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:04.593526 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 404/796] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-0540a07f-9551-42e8-b2a7-dc4b793cecf9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:13:04.611179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:04.612130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:04.613055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:04.613055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:04.613055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:13:04.613055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:13:04.613055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:13:04.613055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:13:04.613055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:13:04.613055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:13:04.613055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:13:04.613055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:04.615389 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 393/797] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Oct 14 02:13:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:04.617933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a9f64-c5d8-48b3-8c95-a579a53264ab None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:04.653100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:04.721250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a9f64-c5d8-48b3-8c95-a579a53264ab None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:04.758591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5fdad-476b-443c-a65c-abc3e53e4d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:04.759233 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5fdad-476b-443c-a65c-abc3e53e4d39 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:04.759535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5fdad-476b-443c-a65c-abc3e53e4d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:04.759806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5fdad-476b-443c-a65c-abc3e53e4d39 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:04.760556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5fdad-476b-443c-a65c-abc3e53e4d39 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:04.760833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5fdad-476b-443c-a65c-abc3e53e4d39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:04.762246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5fdad-476b-443c-a65c-abc3e53e4d39 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:04.781048 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 405/798] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Oct 14 02:13:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:04.786616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540a07f-9551-42e8-b2a7-dc4b793cecf9 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:04.887517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5fdad-476b-443c-a65c-abc3e53e4d39 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:04.901422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540a07f-9551-42e8-b2a7-dc4b793cecf9 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:04.926309 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:04.939219 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cc107-2a53-4540-94a4-f12a8f99124d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:04.939969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cc107-2a53-4540-94a4-f12a8f99124d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:04.940130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cc107-2a53-4540-94a4-f12a8f99124d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:04.940400 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cc107-2a53-4540-94a4-f12a8f99124d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:04.953889 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cc107-2a53-4540-94a4-f12a8f99124d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:04.955993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cc107-2a53-4540-94a4-f12a8f99124d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-245cc107-2a53-4540-94a4-f12a8f99124d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:13:04.971766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:04.972712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:13:04.973577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:13:04.973577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:13:04.973577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:04.973577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 394/799] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:04.978411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cc107-2a53-4540-94a4-f12a8f99124d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:04.989905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:05.001149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5fdad-476b-443c-a65c-abc3e53e4d39 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:05.038584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f6489-6a98-46dd-b5b8-4deb250ecc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:05.039322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f6489-6a98-46dd-b5b8-4deb250ecc24 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:05.039600 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f6489-6a98-46dd-b5b8-4deb250ecc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:05.039870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f6489-6a98-46dd-b5b8-4deb250ecc24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:05.041051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f6489-6a98-46dd-b5b8-4deb250ecc24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:05.042472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f6489-6a98-46dd-b5b8-4deb250ecc24 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:05.044224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 406/800] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:13:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:13:05.049831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f6489-6a98-46dd-b5b8-4deb250ecc24 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:05.092702 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cc107-2a53-4540-94a4-f12a8f99124d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:05.131533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fca89d-418b-475d-b0d4-d67235941ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:05.132154 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fca89d-418b-475d-b0d4-d67235941ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:05.132439 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fca89d-418b-475d-b0d4-d67235941ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:05.132716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fca89d-418b-475d-b0d4-d67235941ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:05.133366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fca89d-418b-475d-b0d4-d67235941ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:05.133674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fca89d-418b-475d-b0d4-d67235941ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:05.135048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fca89d-418b-475d-b0d4-d67235941ee6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:05.148121 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 395/801] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:13:05.153424 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fca89d-418b-475d-b0d4-d67235941ee6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:05.161084 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f6489-6a98-46dd-b5b8-4deb250ecc24 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:05.190848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:05.198589 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae1153-448b-4733-9d88-1324b34080a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:05.199237 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae1153-448b-4733-9d88-1324b34080a7 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:05.199516 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae1153-448b-4733-9d88-1324b34080a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:05.199790 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae1153-448b-4733-9d88-1324b34080a7 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:05.201049 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae1153-448b-4733-9d88-1324b34080a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:05.202487 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae1153-448b-4733-9d88-1324b34080a7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:05.227099 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 407/802] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:13:05 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:13:05.280934 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fca89d-418b-475d-b0d4-d67235941ee6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:05.319649 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e763d-e975-4581-b125-e6ef66ef3ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:05.320364 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e763d-e975-4581-b125-e6ef66ef3ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:05.320650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e763d-e975-4581-b125-e6ef66ef3ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:05.320922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e763d-e975-4581-b125-e6ef66ef3ca8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:05.329204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e763d-e975-4581-b125-e6ef66ef3ca8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:05.330772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e763d-e975-4581-b125-e6ef66ef3ca8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d34e763d-e975-4581-b125-e6ef66ef3ca8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:13:05.343660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:05.344619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:05.350863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:13:05.350863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:13:05.350863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:13:05.350863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:13:05.350863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:13:05.350863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:13:05.350863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:13:05.350863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:05.350863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 396/803] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:13:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:13:05.352492 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae1153-448b-4733-9d88-1324b34080a7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:05.376081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:05.467496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae1153-448b-4733-9d88-1324b34080a7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:05.474472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e372aa-0b41-4015-9b74-9adb2a585c84 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:05.474761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e372aa-0b41-4015-9b74-9adb2a585c84 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:05.475032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e372aa-0b41-4015-9b74-9adb2a585c84 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:05.476524 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 397/804] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:05.481355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63327d9-5df4-4764-acc1-e81d36a063fa None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:05.482171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63327d9-5df4-4764-acc1-e81d36a063fa None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:05.482547 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63327d9-5df4-4764-acc1-e81d36a063fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:05.502731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c839159-ed0e-4815-8087-90b9f061a4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:05.503111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e63327d9-5df4-4764-acc1-e81d36a063fa None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:05.503381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c839159-ed0e-4815-8087-90b9f061a4cd None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:05.503661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c839159-ed0e-4815-8087-90b9f061a4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:05.503933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c839159-ed0e-4815-8087-90b9f061a4cd None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:05.504266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e63327d9-5df4-4764-acc1-e81d36a063fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:05.504692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c839159-ed0e-4815-8087-90b9f061a4cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:05.505001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c839159-ed0e-4815-8087-90b9f061a4cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:05.506430 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c839159-ed0e-4815-8087-90b9f061a4cd None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:05.525026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 408/805] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:13:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:05.526226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c839159-ed0e-4815-8087-90b9f061a4cd None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:05.575509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 398/806] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:13:05.608507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:05.634364 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c839159-ed0e-4815-8087-90b9f061a4cd None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:05.670439 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec5f0a-9c57-4ed4-b196-fb4ccd4de2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:05.671136 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec5f0a-9c57-4ed4-b196-fb4ccd4de2ac None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:05.671420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec5f0a-9c57-4ed4-b196-fb4ccd4de2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:05.671694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec5f0a-9c57-4ed4-b196-fb4ccd4de2ac None admin] PATH_INFO: `/v3/projects/f28d337589024cc88184355bd0ee8572/users/de13be55b4034477baf4c029c1a79757/roles/79e6c6fbf4b446da89c0e323be7e232b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:05.673838 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec5f0a-9c57-4ed4-b196-fb4ccd4de2ac None admin] RBAC: Authorizing `identity:create_grant(project_id=f28d337589024cc88184355bd0ee8572, user_id=de13be55b4034477baf4c029c1a79757, role_id=79e6c6fbf4b446da89c0e323be7e232b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:05.675399 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec5f0a-9c57-4ed4-b196-fb4ccd4de2ac None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:05.695846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 409/807] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:13:05 2022] PUT /identity/v3/projects/f28d337589024cc88184355bd0ee8572/users/de13be55b4034477baf4c029c1a79757/roles/79e6c6fbf4b446da89c0e323be7e232b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:13:05.845355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63327d9-5df4-4764-acc1-e81d36a063fa None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:05.872545 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:05.947408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63327d9-5df4-4764-acc1-e81d36a063fa None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:05.983891 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efb451-079c-4e8b-ae16-8086606f1b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:05.984426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efb451-079c-4e8b-ae16-8086606f1b05 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:05.984705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efb451-079c-4e8b-ae16-8086606f1b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:05.984977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efb451-079c-4e8b-ae16-8086606f1b05 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:05.985658 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efb451-079c-4e8b-ae16-8086606f1b05 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:05.987041 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efb451-079c-4e8b-ae16-8086606f1b05 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:05.995259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.sql.core [None req-f0efb451-079c-4e8b-ae16-8086606f1b05 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 02:13:05.995259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 02:13:05.995259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [parameters: {'id': '62af2a4d0c6d4a30bb0862f7dabedeaf', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 14 02:13:05.995259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81362) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 02:13:06.003196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-f0efb451-079c-4e8b-ae16-8086606f1b05 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 14 02:13:06.003196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:06.003196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:13:06.003196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:13:06.003196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:13:06.003196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:13:06.003196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:13:06.003196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:13:06.003196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:13:06.003196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:13:06.003196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:13:06.003196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:13:06.003196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:13:06.003196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result.read() Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 14 02:13:06.003993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application yield resource Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise self.value Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application session.commit() Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 02:13:06.009793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.session.flush() Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application compat.raise_( Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 02:13:06.010629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 02:13:06.011537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result.read() Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:13:06.012684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [parameters: {'id': '62af2a4d0c6d4a30bb0862f7dabedeaf', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 14 02:13:06.014532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:06.016299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:06.018394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 14 02:13:06.018394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 14 02:13:06.018394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:06.018394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:06.018394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 14 02:13:06.018394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 14 02:13:06.018394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:06.018394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:06.018394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 14 02:13:06.018394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 14 02:13:06.018394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 14 02:13:06.018394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:06.018394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 399/808] 10.222.0.18 () {60 vars in 1137 bytes} [Fri Oct 14 02:13:05 2022] POST /identity/v3/domains => generated 190 bytes in 160 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:06.019378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec5f0a-9c57-4ed4-b196-fb4ccd4de2ac None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:06.122063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec5f0a-9c57-4ed4-b196-fb4ccd4de2ac None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:06.157309 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df9d45-75b8-45c7-bb62-d548274e1095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:06.158053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df9d45-75b8-45c7-bb62-d548274e1095 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:06.158332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df9d45-75b8-45c7-bb62-d548274e1095 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:06.158599 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df9d45-75b8-45c7-bb62-d548274e1095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:06.161953 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:06.174022 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df9d45-75b8-45c7-bb62-d548274e1095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:06.175560 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df9d45-75b8-45c7-bb62-d548274e1095 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-79df9d45-75b8-45c7-bb62-d548274e1095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:13:06.190341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:06.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:13:06.197463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:13:06.197463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:13:06.197463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:13:06.197463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 410/809] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:06.197685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efb451-079c-4e8b-ae16-8086606f1b05 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:06.287754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:06.298901 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efb451-079c-4e8b-ae16-8086606f1b05 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:06.332759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcd941-ab1e-4c07-8003-561b17f1053f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:06.333372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcd941-ab1e-4c07-8003-561b17f1053f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:06.333650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcd941-ab1e-4c07-8003-561b17f1053f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:06.333923 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcd941-ab1e-4c07-8003-561b17f1053f None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:06.334560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcd941-ab1e-4c07-8003-561b17f1053f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:06.334860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcd941-ab1e-4c07-8003-561b17f1053f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:06.336240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcd941-ab1e-4c07-8003-561b17f1053f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:06.348264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 400/810] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:13:07.428878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b45f9-4b07-442c-af25-1d26ccb492f2 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:07.429460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b45f9-4b07-442c-af25-1d26ccb492f2 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:07.429460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b45f9-4b07-442c-af25-1d26ccb492f2 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:07.430741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 411/811] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:07.436733 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d67e76-25c5-41cf-aef7-188fbf81a407 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:07.437151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d67e76-25c5-41cf-aef7-188fbf81a407 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:07.437341 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d67e76-25c5-41cf-aef7-188fbf81a407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:07.455573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c3d67e76-25c5-41cf-aef7-188fbf81a407 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:07.456738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c3d67e76-25c5-41cf-aef7-188fbf81a407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:07.522728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 401/812] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:07.788215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b45f9-4b07-442c-af25-1d26ccb492f2 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:07.858498 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f4f27-2e49-4ea4-bbb9-0c177d7d75b7 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:07.858771 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f4f27-2e49-4ea4-bbb9-0c177d7d75b7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:07.859038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f4f27-2e49-4ea4-bbb9-0c177d7d75b7 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:07.860379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 402/813] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:07.865216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa54b2-097d-49d0-8dcd-5f6939ec6141 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:07.865503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa54b2-097d-49d0-8dcd-5f6939ec6141 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:07.865764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa54b2-097d-49d0-8dcd-5f6939ec6141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:07.883728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e6fa54b2-097d-49d0-8dcd-5f6939ec6141 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:07.884904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e6fa54b2-097d-49d0-8dcd-5f6939ec6141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:07.888150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b45f9-4b07-442c-af25-1d26ccb492f2 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:07.921932 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74743f1-1498-4ec6-88e2-74d574d0fa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:07.922559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74743f1-1498-4ec6-88e2-74d574d0fa7e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:07.922838 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74743f1-1498-4ec6-88e2-74d574d0fa7e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:07.923140 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74743f1-1498-4ec6-88e2-74d574d0fa7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:07.933210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74743f1-1498-4ec6-88e2-74d574d0fa7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-b74743f1-1498-4ec6-88e2-74d574d0fa7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:13:07.937986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:07.938982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:07.939990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:07.939990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:07.939990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:07.939990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:07.939990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:13:07.939990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:13:07.939990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:13:07.939990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:13:07.939990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:13:07.939990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:13:07.939990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:13:07.939990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:13:07.939990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 412/814] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:13:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:07.944960 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74743f1-1498-4ec6-88e2-74d574d0fa7e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:07.949945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 403/815] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:13:07.992850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:08.042290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74743f1-1498-4ec6-88e2-74d574d0fa7e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:08.074943 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbd1a7-94ac-47ff-aa0b-8fb400545a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:08.076279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbd1a7-94ac-47ff-aa0b-8fb400545a1a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:08.076565 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbd1a7-94ac-47ff-aa0b-8fb400545a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:08.076844 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbd1a7-94ac-47ff-aa0b-8fb400545a1a None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:08.077506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbd1a7-94ac-47ff-aa0b-8fb400545a1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:08.077808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbd1a7-94ac-47ff-aa0b-8fb400545a1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:08.079189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbd1a7-94ac-47ff-aa0b-8fb400545a1a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:08.087353 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 413/816] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:13:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:13:08.092890 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa54b2-097d-49d0-8dcd-5f6939ec6141 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:08.104452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:08.189532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa54b2-097d-49d0-8dcd-5f6939ec6141 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:08.216513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbd1a7-94ac-47ff-aa0b-8fb400545a1a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:08.222456 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34406782-0fdc-407b-90f0-2d716c541539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:08.223111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34406782-0fdc-407b-90f0-2d716c541539 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:08.223389 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34406782-0fdc-407b-90f0-2d716c541539 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:08.223656 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34406782-0fdc-407b-90f0-2d716c541539 None admin] PATH_INFO: `/v3/users/fad0d3cccf9a49cabcfc5ec830e107af` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:08.237428 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34406782-0fdc-407b-90f0-2d716c541539 None admin] RBAC: Authorizing `identity:get_user(user_id=fad0d3cccf9a49cabcfc5ec830e107af)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:08.238861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34406782-0fdc-407b-90f0-2d716c541539 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:08.240490 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 404/817] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:13:08 2022] GET /identity/v3/users/fad0d3cccf9a49cabcfc5ec830e107af => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:08.246298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34406782-0fdc-407b-90f0-2d716c541539 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:08.310927 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:08.315948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbd1a7-94ac-47ff-aa0b-8fb400545a1a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:08.326003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:08.344485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34406782-0fdc-407b-90f0-2d716c541539 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:08.350314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abf9f0-9fc8-4b06-8b45-3fec142e3583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:08.350941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abf9f0-9fc8-4b06-8b45-3fec142e3583 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:08.351251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abf9f0-9fc8-4b06-8b45-3fec142e3583 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:08.351526 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abf9f0-9fc8-4b06-8b45-3fec142e3583 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:08.362530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abf9f0-9fc8-4b06-8b45-3fec142e3583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a4abf9f0-9fc8-4b06-8b45-3fec142e3583 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:13:08.367261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:08.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:08.369283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:08.369283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:08.369283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:08.369283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:08.369283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:13:08.369283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:13:08.369283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:13:08.369283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:13:08.369283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:13:08.369283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:13:08.369283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:13:08.369283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:13:08.369283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 414/818] 10.222.0.18 () {56 vars in 1120 bytes} [Fri Oct 14 02:13:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:13:08.375111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abf9f0-9fc8-4b06-8b45-3fec142e3583 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:08.379133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47718963-5c29-4953-9459-58932ff12343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:08.379774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47718963-5c29-4953-9459-58932ff12343 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:08.380054 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47718963-5c29-4953-9459-58932ff12343 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:08.380330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47718963-5c29-4953-9459-58932ff12343 None admin] PATH_INFO: `/v3/projects/e37e85a831ab428d8332d7c1979e8e10` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:08.381442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47718963-5c29-4953-9459-58932ff12343 None admin] RBAC: Authorizing `identity:get_project(project_id=e37e85a831ab428d8332d7c1979e8e10)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:08.382881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47718963-5c29-4953-9459-58932ff12343 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:08.384581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 405/819] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:13:08 2022] GET /identity/v3/projects/e37e85a831ab428d8332d7c1979e8e10 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:13:08.391105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47718963-5c29-4953-9459-58932ff12343 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:08.476852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abf9f0-9fc8-4b06-8b45-3fec142e3583 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:08.495286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47718963-5c29-4953-9459-58932ff12343 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:08.509445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129622c9-bc1a-4101-845c-3c8e8b95968b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:08.510039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129622c9-bc1a-4101-845c-3c8e8b95968b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:08.510316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129622c9-bc1a-4101-845c-3c8e8b95968b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:08.510585 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129622c9-bc1a-4101-845c-3c8e8b95968b None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:08.511269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129622c9-bc1a-4101-845c-3c8e8b95968b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:08.511572 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129622c9-bc1a-4101-845c-3c8e8b95968b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:08.513561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129622c9-bc1a-4101-845c-3c8e8b95968b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:08.521220 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 415/820] 10.222.0.18 () {56 vars in 1115 bytes} [Fri Oct 14 02:13:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:13:08.526916 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129622c9-bc1a-4101-845c-3c8e8b95968b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:08.529681 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcf990-4902-4242-a615-f9bbbe294237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:08.530349 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcf990-4902-4242-a615-f9bbbe294237 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:08.530630 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcf990-4902-4242-a615-f9bbbe294237 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:08.530912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcf990-4902-4242-a615-f9bbbe294237 None admin] PATH_INFO: `/v3/projects/e37e85a831ab428d8332d7c1979e8e10/users/fad0d3cccf9a49cabcfc5ec830e107af/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:08.532841 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fcf990-4902-4242-a615-f9bbbe294237 None admin] RBAC: Authorizing `identity:create_grant(project_id=e37e85a831ab428d8332d7c1979e8e10, user_id=fad0d3cccf9a49cabcfc5ec830e107af, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:08.534266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fcf990-4902-4242-a615-f9bbbe294237 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:08.537404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:08.548802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 406/821] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:13:08 2022] PUT /identity/v3/projects/e37e85a831ab428d8332d7c1979e8e10/users/fad0d3cccf9a49cabcfc5ec830e107af/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:13:08.559383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:08.633394 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129622c9-bc1a-4101-845c-3c8e8b95968b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:08.669863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4183252-443a-4da9-bda7-69d5f7968f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:08.670528 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4183252-443a-4da9-bda7-69d5f7968f02 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:08.670807 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4183252-443a-4da9-bda7-69d5f7968f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:08.671095 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4183252-443a-4da9-bda7-69d5f7968f02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:08.682522 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4183252-443a-4da9-bda7-69d5f7968f02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:08.684031 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4183252-443a-4da9-bda7-69d5f7968f02 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-f4183252-443a-4da9-bda7-69d5f7968f02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:13:08.699630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:08.701254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:13:08.702733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:13:08.702733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:13:08.702733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:13:08.702733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 416/822] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:13:08.707393 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcf990-4902-4242-a615-f9bbbe294237 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:08.807864 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcf990-4902-4242-a615-f9bbbe294237 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:08.841866 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8bb39-9620-4d46-af96-d0d44d125b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:08.842482 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8bb39-9620-4d46-af96-d0d44d125b07 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:08.842767 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8bb39-9620-4d46-af96-d0d44d125b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:08.843041 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8bb39-9620-4d46-af96-d0d44d125b07 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:08.843709 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8bb39-9620-4d46-af96-d0d44d125b07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:08.844013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8bb39-9620-4d46-af96-d0d44d125b07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:08.845388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8bb39-9620-4d46-af96-d0d44d125b07 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:08.857564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 407/823] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:08.863206 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4183252-443a-4da9-bda7-69d5f7968f02 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:08.949496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:08.966189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4183252-443a-4da9-bda7-69d5f7968f02 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:08.999693 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181edcb7-abfe-41b7-a02e-39a56a80ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:09.000323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181edcb7-abfe-41b7-a02e-39a56a80ac4d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:09.000603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181edcb7-abfe-41b7-a02e-39a56a80ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:09.000877 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181edcb7-abfe-41b7-a02e-39a56a80ac4d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:09.008175 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181edcb7-abfe-41b7-a02e-39a56a80ac4d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:09.009605 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181edcb7-abfe-41b7-a02e-39a56a80ac4d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-181edcb7-abfe-41b7-a02e-39a56a80ac4d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:13:09.020715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:09.021774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:09.022731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:09.022731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:09.022731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:13:09.022731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:13:09.022731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:13:09.022731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:13:09.022731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:13:09.022731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:13:09.022731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:13:09.022731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:13:09.022731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 417/824] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Oct 14 02:13:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:09.027575 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8bb39-9620-4d46-af96-d0d44d125b07 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:09.083873 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:09.240630 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8bb39-9620-4d46-af96-d0d44d125b07 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:09.274208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1830fab-5cb7-4ba9-a243-9fa478e9a8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:09.274818 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1830fab-5cb7-4ba9-a243-9fa478e9a8de None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:09.275121 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1830fab-5cb7-4ba9-a243-9fa478e9a8de None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:09.275399 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1830fab-5cb7-4ba9-a243-9fa478e9a8de None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:09.276127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1830fab-5cb7-4ba9-a243-9fa478e9a8de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:09.276432 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1830fab-5cb7-4ba9-a243-9fa478e9a8de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:09.277872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1830fab-5cb7-4ba9-a243-9fa478e9a8de None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:09.299371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 408/825] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Oct 14 02:13:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:09.300713 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181edcb7-abfe-41b7-a02e-39a56a80ac4d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:09.397318 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181edcb7-abfe-41b7-a02e-39a56a80ac4d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:09.433281 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64030567-d07e-4e13-9550-6677396c53e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:09.433892 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64030567-d07e-4e13-9550-6677396c53e0 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:09.434171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64030567-d07e-4e13-9550-6677396c53e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:09.434460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64030567-d07e-4e13-9550-6677396c53e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:09.449051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:09.502841 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64030567-d07e-4e13-9550-6677396c53e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:09.504339 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64030567-d07e-4e13-9550-6677396c53e0 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:09.601415 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 418/826] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:13:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:13:09.607744 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1830fab-5cb7-4ba9-a243-9fa478e9a8de None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:09.703193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1830fab-5cb7-4ba9-a243-9fa478e9a8de None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:09.716447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:09.744273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf4e50-ae31-40d0-92cd-f12c7d4efe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:09.744920 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf4e50-ae31-40d0-92cd-f12c7d4efe27 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:09.745200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf4e50-ae31-40d0-92cd-f12c7d4efe27 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:09.745472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf4e50-ae31-40d0-92cd-f12c7d4efe27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:09.756349 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf4e50-ae31-40d0-92cd-f12c7d4efe27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:09.757808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf4e50-ae31-40d0-92cd-f12c7d4efe27 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-1aaf4e50-ae31-40d0-92cd-f12c7d4efe27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:13:09.772746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:09.773848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:13:09.774735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:13:09.774735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:13:09.774735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:09.774735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 409/827] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:13:09.779669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64030567-d07e-4e13-9550-6677396c53e0 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:09.823355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:09.876635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64030567-d07e-4e13-9550-6677396c53e0 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:09.909419 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd5032-0a1f-45ae-84ef-dee22475bc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:09.910022 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd5032-0a1f-45ae-84ef-dee22475bc4c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:09.910299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd5032-0a1f-45ae-84ef-dee22475bc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:09.910570 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd5032-0a1f-45ae-84ef-dee22475bc4c None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:09.911243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd5032-0a1f-45ae-84ef-dee22475bc4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:09.911545 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd5032-0a1f-45ae-84ef-dee22475bc4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:09.912895 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd5032-0a1f-45ae-84ef-dee22475bc4c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:09.923601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 419/828] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:09.929335 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf4e50-ae31-40d0-92cd-f12c7d4efe27 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:10.026885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf4e50-ae31-40d0-92cd-f12c7d4efe27 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:10.059175 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758c7b5-b554-4175-8ea2-8cecce02a1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:10.060371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758c7b5-b554-4175-8ea2-8cecce02a1cb None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:10.060648 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758c7b5-b554-4175-8ea2-8cecce02a1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:10.060923 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758c7b5-b554-4175-8ea2-8cecce02a1cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:10.067393 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758c7b5-b554-4175-8ea2-8cecce02a1cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:10.068835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758c7b5-b554-4175-8ea2-8cecce02a1cb None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-1758c7b5-b554-4175-8ea2-8cecce02a1cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:13:10.079965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:10.080948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:10.081799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:13:10.081799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:13:10.081799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:13:10.081799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:13:10.081799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:13:10.081799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:13:10.081799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:13:10.081799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:10.081799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 410/829] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:13:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:10.086796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd5032-0a1f-45ae-84ef-dee22475bc4c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:10.182670 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd5032-0a1f-45ae-84ef-dee22475bc4c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:10.185624 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:10.216173 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b08655-39af-4f0a-bfdd-f866d02ee62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:10.216774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b08655-39af-4f0a-bfdd-f866d02ee62b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:10.217049 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b08655-39af-4f0a-bfdd-f866d02ee62b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:10.217316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b08655-39af-4f0a-bfdd-f866d02ee62b None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:10.217997 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b08655-39af-4f0a-bfdd-f866d02ee62b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:10.218297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b08655-39af-4f0a-bfdd-f866d02ee62b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:10.219739 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b08655-39af-4f0a-bfdd-f866d02ee62b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:10.230695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 420/830] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:13:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:10.237307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758c7b5-b554-4175-8ea2-8cecce02a1cb None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:10.265073 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:10.336739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758c7b5-b554-4175-8ea2-8cecce02a1cb None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:10.369150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe234bb-e116-4897-93b6-7b62b8b7167c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:10.369760 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe234bb-e116-4897-93b6-7b62b8b7167c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:10.370043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe234bb-e116-4897-93b6-7b62b8b7167c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:10.370318 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe234bb-e116-4897-93b6-7b62b8b7167c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:10.371339 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe234bb-e116-4897-93b6-7b62b8b7167c None admin] RBAC: Adding query filter params (role.id=79e6c6fbf4b446da89c0e323be7e232b, scope.project.id=f28d337589024cc88184355bd0ee8572, user.id=de13be55b4034477baf4c029c1a79757) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:10.371651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe234bb-e116-4897-93b6-7b62b8b7167c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:10.373634 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe234bb-e116-4897-93b6-7b62b8b7167c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:10.379237 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 411/831] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:13:10 2022] GET /identity/v3/role_assignments?user.id=de13be55b4034477baf4c029c1a79757&scope.project.id=f28d337589024cc88184355bd0ee8572&role.id=79e6c6fbf4b446da89c0e323be7e232b => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:10.685800 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082ef46-d0ee-49ab-9f3e-16c0ec8f79bd None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:10.686283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082ef46-d0ee-49ab-9f3e-16c0ec8f79bd None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:10.686283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082ef46-d0ee-49ab-9f3e-16c0ec8f79bd None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:10.687603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 421/832] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:10.693131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863b881-39ee-43d9-a377-f0593ebcdff9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:10.693576 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863b881-39ee-43d9-a377-f0593ebcdff9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:10.693682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863b881-39ee-43d9-a377-f0593ebcdff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:10.713822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9863b881-39ee-43d9-a377-f0593ebcdff9 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:10.715024 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9863b881-39ee-43d9-a377-f0593ebcdff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:10.784697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 412/833] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:11.056447 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082ef46-d0ee-49ab-9f3e-16c0ec8f79bd None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:11.162054 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082ef46-d0ee-49ab-9f3e-16c0ec8f79bd None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:11.196337 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ba451-3d77-4c1c-bd1c-4f34d94eeb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:11.196968 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ba451-3d77-4c1c-bd1c-4f34d94eeb9b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:11.197244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ba451-3d77-4c1c-bd1c-4f34d94eeb9b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:11.197515 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ba451-3d77-4c1c-bd1c-4f34d94eeb9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:11.209087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ba451-3d77-4c1c-bd1c-4f34d94eeb9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-c27ba451-3d77-4c1c-bd1c-4f34d94eeb9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:13:11.213865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:11.214777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:11.216402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:11.216402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:11.216402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:11.216402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:11.216402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:13:11.216402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:13:11.216402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:13:11.216402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:13:11.216402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:13:11.216402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:13:11.216402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:13:11.216402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:13:11.216402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 422/834] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:13:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:11.221687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863b881-39ee-43d9-a377-f0593ebcdff9 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:11.242523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:11.317805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863b881-39ee-43d9-a377-f0593ebcdff9 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:11.350113 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2d847-20dd-42f4-9423-3c078a822a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:11.350714 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2d847-20dd-42f4-9423-3c078a822a88 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:11.350993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2d847-20dd-42f4-9423-3c078a822a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:11.351295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2d847-20dd-42f4-9423-3c078a822a88 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:11.351954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2d847-20dd-42f4-9423-3c078a822a88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:11.352258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2d847-20dd-42f4-9423-3c078a822a88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:11.353608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2d847-20dd-42f4-9423-3c078a822a88 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:11.361497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 413/835] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:13:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:11.367398 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ba451-3d77-4c1c-bd1c-4f34d94eeb9b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:11.393551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:11.464965 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ba451-3d77-4c1c-bd1c-4f34d94eeb9b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:11.498586 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91541af2-f6c8-4ee6-97cd-6f2ae99629c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:11.499229 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91541af2-f6c8-4ee6-97cd-6f2ae99629c7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:11.499505 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91541af2-f6c8-4ee6-97cd-6f2ae99629c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:11.499775 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91541af2-f6c8-4ee6-97cd-6f2ae99629c7 None admin] PATH_INFO: `/v3/users/fad0d3cccf9a49cabcfc5ec830e107af` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:11.500862 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91541af2-f6c8-4ee6-97cd-6f2ae99629c7 None admin] RBAC: Authorizing `identity:get_user(user_id=fad0d3cccf9a49cabcfc5ec830e107af)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:11.502286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91541af2-f6c8-4ee6-97cd-6f2ae99629c7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:11.503847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 423/836] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:13:11 2022] GET /identity/v3/users/fad0d3cccf9a49cabcfc5ec830e107af => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:11.509516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2d847-20dd-42f4-9423-3c078a822a88 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:11.586689 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:11.604158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2d847-20dd-42f4-9423-3c078a822a88 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:11.636527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f0e26-64e8-4aaf-8ec5-f7cb7b6ab207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:11.637116 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f0e26-64e8-4aaf-8ec5-f7cb7b6ab207 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:11.637419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f0e26-64e8-4aaf-8ec5-f7cb7b6ab207 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:11.637686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f0e26-64e8-4aaf-8ec5-f7cb7b6ab207 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:11.700168 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f0e26-64e8-4aaf-8ec5-f7cb7b6ab207 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:11.701581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f0e26-64e8-4aaf-8ec5-f7cb7b6ab207 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:11.776990 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:11.796022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 414/837] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:13:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:13:11.801868 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91541af2-f6c8-4ee6-97cd-6f2ae99629c7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:11.895060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:11.903926 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91541af2-f6c8-4ee6-97cd-6f2ae99629c7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:11.936751 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc7418-aab2-4cd1-becb-963c9efb6a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:11.937372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc7418-aab2-4cd1-becb-963c9efb6a3f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:11.937646 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc7418-aab2-4cd1-becb-963c9efb6a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:11.937914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc7418-aab2-4cd1-becb-963c9efb6a3f None admin] PATH_INFO: `/v3/projects/e37e85a831ab428d8332d7c1979e8e10` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:11.938996 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc7418-aab2-4cd1-becb-963c9efb6a3f None admin] RBAC: Authorizing `identity:get_project(project_id=e37e85a831ab428d8332d7c1979e8e10)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:11.940432 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc7418-aab2-4cd1-becb-963c9efb6a3f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:11.942010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 424/838] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:13:11 2022] GET /identity/v3/projects/e37e85a831ab428d8332d7c1979e8e10 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:11.948000 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f0e26-64e8-4aaf-8ec5-f7cb7b6ab207 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:12.040510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f0e26-64e8-4aaf-8ec5-f7cb7b6ab207 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:12.072437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c92883-fd72-4d8f-b69f-f41d2bac585b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:12.073024 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c92883-fd72-4d8f-b69f-f41d2bac585b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:12.073298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c92883-fd72-4d8f-b69f-f41d2bac585b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:12.073563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c92883-fd72-4d8f-b69f-f41d2bac585b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:12.074264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c92883-fd72-4d8f-b69f-f41d2bac585b None admin] RBAC: Adding query filter params (role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b, scope.project.id=e37e85a831ab428d8332d7c1979e8e10, user.id=fad0d3cccf9a49cabcfc5ec830e107af) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:12.074564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c92883-fd72-4d8f-b69f-f41d2bac585b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:12.076138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c92883-fd72-4d8f-b69f-f41d2bac585b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:12.081592 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 415/839] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:13:11 2022] GET /identity/v3/role_assignments?user.id=fad0d3cccf9a49cabcfc5ec830e107af&scope.project.id=e37e85a831ab428d8332d7c1979e8e10&role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:14.213980 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6aac2-648a-4de2-bdfd-9be5bcb4ad3d None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:14.214543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6aac2-648a-4de2-bdfd-9be5bcb4ad3d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:14.214543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6aac2-648a-4de2-bdfd-9be5bcb4ad3d None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:14.216041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 425/840] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:14.221327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8163b07-03ab-474b-8533-fd678bce3133 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:14.221731 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8163b07-03ab-474b-8533-fd678bce3133 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:14.221867 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8163b07-03ab-474b-8533-fd678bce3133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:14.239577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a8163b07-03ab-474b-8533-fd678bce3133 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:14.240746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a8163b07-03ab-474b-8533-fd678bce3133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:14.282127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:14.305397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 416/841] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:14.572519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6aac2-648a-4de2-bdfd-9be5bcb4ad3d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:14.670357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6aac2-648a-4de2-bdfd-9be5bcb4ad3d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:14.702663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88eb0d-e655-4938-8105-4fbb59060792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:14.703299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88eb0d-e655-4938-8105-4fbb59060792 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:14.703579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88eb0d-e655-4938-8105-4fbb59060792 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:14.703850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88eb0d-e655-4938-8105-4fbb59060792 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:14.704937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88eb0d-e655-4938-8105-4fbb59060792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:14.706350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88eb0d-e655-4938-8105-4fbb59060792 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:14.707908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 426/842] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:13:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:14.713522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8163b07-03ab-474b-8533-fd678bce3133 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:14.809851 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8163b07-03ab-474b-8533-fd678bce3133 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:14.842281 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d783d0-6586-43cf-8f7d-1734b2fdc090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:14.842826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d783d0-6586-43cf-8f7d-1734b2fdc090 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:14.843145 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d783d0-6586-43cf-8f7d-1734b2fdc090 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:14.843423 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d783d0-6586-43cf-8f7d-1734b2fdc090 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:14.844060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d783d0-6586-43cf-8f7d-1734b2fdc090 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:14.845479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d783d0-6586-43cf-8f7d-1734b2fdc090 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:14.866898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 417/843] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:13:14 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:13:17.018764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49d955-db3a-46e4-9ad3-8866431d79e8 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:17.019299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49d955-db3a-46e4-9ad3-8866431d79e8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:17.019388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49d955-db3a-46e4-9ad3-8866431d79e8 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:17.020686 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 427/844] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:17.025931 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccc5b1-03c7-4a29-9919-30cd01060b24 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:17.026317 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccc5b1-03c7-4a29-9919-30cd01060b24 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:17.026478 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccc5b1-03c7-4a29-9919-30cd01060b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:17.044821 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-84ccc5b1-03c7-4a29-9919-30cd01060b24 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:17.046003 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-84ccc5b1-03c7-4a29-9919-30cd01060b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:17.070368 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:17.111130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 418/845] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:17.378485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49d955-db3a-46e4-9ad3-8866431d79e8 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:17.406316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:17.477657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49d955-db3a-46e4-9ad3-8866431d79e8 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:17.510549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc390c-16ca-451a-9069-389c8e26b863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:17.511195 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc390c-16ca-451a-9069-389c8e26b863 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:17.511471 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc390c-16ca-451a-9069-389c8e26b863 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:17.511740 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc390c-16ca-451a-9069-389c8e26b863 None admin] PATH_INFO: `/v3/roles/a3287688078248ecb19a6d9c85839631` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:17.513722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc390c-16ca-451a-9069-389c8e26b863 None admin] RBAC: Authorizing `identity:get_role(role_id=a3287688078248ecb19a6d9c85839631)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:17.515090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc390c-16ca-451a-9069-389c8e26b863 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:17.516396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 428/846] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:13:17 2022] GET /identity/v3/roles/a3287688078248ecb19a6d9c85839631 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:17.522171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccc5b1-03c7-4a29-9919-30cd01060b24 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:17.615803 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccc5b1-03c7-4a29-9919-30cd01060b24 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:17.647955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b089b2-fe66-4c66-894c-0dcf79c1d88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:17.648516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b089b2-fe66-4c66-894c-0dcf79c1d88a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:17.648787 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b089b2-fe66-4c66-894c-0dcf79c1d88a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:17.649053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b089b2-fe66-4c66-894c-0dcf79c1d88a None admin] PATH_INFO: `/v3/users/f691292283544716a73154274e30dc15` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:17.661919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b089b2-fe66-4c66-894c-0dcf79c1d88a None admin] RBAC: Authorizing `identity:get_user(user_id=f691292283544716a73154274e30dc15)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:17.663351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b089b2-fe66-4c66-894c-0dcf79c1d88a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:17.664879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 419/847] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:13:17 2022] GET /identity/v3/users/f691292283544716a73154274e30dc15 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:17.670823 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc390c-16ca-451a-9069-389c8e26b863 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:17.764640 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:17.770603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc390c-16ca-451a-9069-389c8e26b863 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:17.803609 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c359118-15f2-4e31-8f28-a26329737bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:17.804230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c359118-15f2-4e31-8f28-a26329737bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:17.804511 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c359118-15f2-4e31-8f28-a26329737bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:17.804783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c359118-15f2-4e31-8f28-a26329737bd0 None admin] PATH_INFO: `/v3/projects/e37e85a831ab428d8332d7c1979e8e10` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:17.805880 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c359118-15f2-4e31-8f28-a26329737bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=e37e85a831ab428d8332d7c1979e8e10)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:17.807322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c359118-15f2-4e31-8f28-a26329737bd0 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:17.809391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 429/848] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:13:17 2022] GET /identity/v3/projects/e37e85a831ab428d8332d7c1979e8e10 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:17.814922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b089b2-fe66-4c66-894c-0dcf79c1d88a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:17.836008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:17.910509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b089b2-fe66-4c66-894c-0dcf79c1d88a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:17.942460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a8d48-5780-4420-9647-e93f962ba2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:17.943126 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a8d48-5780-4420-9647-e93f962ba2ef None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:17.943406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a8d48-5780-4420-9647-e93f962ba2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:17.943675 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a8d48-5780-4420-9647-e93f962ba2ef None admin] PATH_INFO: `/v3/projects/e37e85a831ab428d8332d7c1979e8e10/users/f691292283544716a73154274e30dc15/roles/a3287688078248ecb19a6d9c85839631` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:17.945737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a8d48-5780-4420-9647-e93f962ba2ef None admin] RBAC: Authorizing `identity:create_grant(project_id=e37e85a831ab428d8332d7c1979e8e10, user_id=f691292283544716a73154274e30dc15, role_id=a3287688078248ecb19a6d9c85839631)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:17.947172 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a8d48-5780-4420-9647-e93f962ba2ef None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:17.961085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 420/849] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:13:17 2022] PUT /identity/v3/projects/e37e85a831ab428d8332d7c1979e8e10/users/f691292283544716a73154274e30dc15/roles/a3287688078248ecb19a6d9c85839631 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:13:20.073139 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc579e5-7883-44c7-b2e6-c05261ab70e5 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:20.073660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc579e5-7883-44c7-b2e6-c05261ab70e5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:20.073660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc579e5-7883-44c7-b2e6-c05261ab70e5 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:20.075569 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 430/850] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:20.080649 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229554e-69bf-4871-b11c-7e4ec4608f70 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:20.081053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229554e-69bf-4871-b11c-7e4ec4608f70 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:20.081189 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229554e-69bf-4871-b11c-7e4ec4608f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:20.099300 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1229554e-69bf-4871-b11c-7e4ec4608f70 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:20.100466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1229554e-69bf-4871-b11c-7e4ec4608f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:20.162859 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 421/851] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:20.440010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc579e5-7883-44c7-b2e6-c05261ab70e5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:20.538833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc579e5-7883-44c7-b2e6-c05261ab70e5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:20.571829 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5e2d1-a876-4f0a-a1f3-664d686920df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:20.572443 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5e2d1-a876-4f0a-a1f3-664d686920df None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:20.572719 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5e2d1-a876-4f0a-a1f3-664d686920df None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:20.572989 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5e2d1-a876-4f0a-a1f3-664d686920df None admin] PATH_INFO: `/v3/roles/a3287688078248ecb19a6d9c85839631` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:20.574835 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5e2d1-a876-4f0a-a1f3-664d686920df None admin] RBAC: Authorizing `identity:get_role(role_id=a3287688078248ecb19a6d9c85839631)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:20.576221 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5e2d1-a876-4f0a-a1f3-664d686920df None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:20.577515 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 431/852] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:13:20 2022] GET /identity/v3/roles/a3287688078248ecb19a6d9c85839631 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:20.583164 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229554e-69bf-4871-b11c-7e4ec4608f70 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:20.594911 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:20.678733 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229554e-69bf-4871-b11c-7e4ec4608f70 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:20.710617 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53d16b-4bca-4406-8eac-4764dbac193b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:20.711199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53d16b-4bca-4406-8eac-4764dbac193b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:20.711474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53d16b-4bca-4406-8eac-4764dbac193b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:20.711741 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53d16b-4bca-4406-8eac-4764dbac193b None admin] PATH_INFO: `/v3/users/f691292283544716a73154274e30dc15` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:20.712838 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53d16b-4bca-4406-8eac-4764dbac193b None admin] RBAC: Authorizing `identity:get_user(user_id=f691292283544716a73154274e30dc15)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:20.714241 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53d16b-4bca-4406-8eac-4764dbac193b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:20.716372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 422/853] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:13:20 2022] GET /identity/v3/users/f691292283544716a73154274e30dc15 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:20.722129 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5e2d1-a876-4f0a-a1f3-664d686920df None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:20.753188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:20.818950 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5e2d1-a876-4f0a-a1f3-664d686920df None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:20.851382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d282a13-7704-429e-88a2-83a830b8280d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:20.851976 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d282a13-7704-429e-88a2-83a830b8280d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:20.852251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d282a13-7704-429e-88a2-83a830b8280d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:20.852519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d282a13-7704-429e-88a2-83a830b8280d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:20.916352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d282a13-7704-429e-88a2-83a830b8280d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:20.917775 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d282a13-7704-429e-88a2-83a830b8280d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:20.948665 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:21.017256 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 432/854] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:13:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:13:21.023317 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53d16b-4bca-4406-8eac-4764dbac193b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:21.097733 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:21.118466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53d16b-4bca-4406-8eac-4764dbac193b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:21.150127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a11f9-2bb4-4042-a485-2c669b06e5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:21.150739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a11f9-2bb4-4042-a485-2c669b06e5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:21.151011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a11f9-2bb4-4042-a485-2c669b06e5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:21.151304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a11f9-2bb4-4042-a485-2c669b06e5c5 None admin] PATH_INFO: `/v3/projects/e37e85a831ab428d8332d7c1979e8e10` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:21.152923 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a11f9-2bb4-4042-a485-2c669b06e5c5 None admin] RBAC: Authorizing `identity:get_project(project_id=e37e85a831ab428d8332d7c1979e8e10)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:21.154326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a11f9-2bb4-4042-a485-2c669b06e5c5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:21.155939 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 423/855] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:13:21 2022] GET /identity/v3/projects/e37e85a831ab428d8332d7c1979e8e10 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:21.162229 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d282a13-7704-429e-88a2-83a830b8280d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:21.256421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d282a13-7704-429e-88a2-83a830b8280d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:21.284228 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:21.289372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa2fc7-dcaa-48e8-aab8-b51708c7b807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:21.289960 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa2fc7-dcaa-48e8-aab8-b51708c7b807 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:21.290235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa2fc7-dcaa-48e8-aab8-b51708c7b807 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:21.290503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa2fc7-dcaa-48e8-aab8-b51708c7b807 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:21.291241 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa2fc7-dcaa-48e8-aab8-b51708c7b807 None admin] RBAC: Adding query filter params (role.id=a3287688078248ecb19a6d9c85839631, scope.project.id=e37e85a831ab428d8332d7c1979e8e10, user.id=f691292283544716a73154274e30dc15) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:21.291544 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa2fc7-dcaa-48e8-aab8-b51708c7b807 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:21.293608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa2fc7-dcaa-48e8-aab8-b51708c7b807 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:21.299360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 433/856] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:13:21 2022] GET /identity/v3/role_assignments?user.id=f691292283544716a73154274e30dc15&scope.project.id=e37e85a831ab428d8332d7c1979e8e10&role.id=a3287688078248ecb19a6d9c85839631 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:23.414245 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2220cb-c22a-4d30-8ce2-0da4847710d7 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:23.414797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2220cb-c22a-4d30-8ce2-0da4847710d7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:23.414797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2220cb-c22a-4d30-8ce2-0da4847710d7 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:23.416149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 424/857] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:23.421505 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0527369-5241-4c96-90bc-afc68673a26c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:23.421922 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0527369-5241-4c96-90bc-afc68673a26c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:23.422043 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0527369-5241-4c96-90bc-afc68673a26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:23.440063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a0527369-5241-4c96-90bc-afc68673a26c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:23.441217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a0527369-5241-4c96-90bc-afc68673a26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:23.504965 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 434/858] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:23.781648 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2220cb-c22a-4d30-8ce2-0da4847710d7 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:23.879072 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2220cb-c22a-4d30-8ce2-0da4847710d7 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:23.910985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca3529-8ed0-4ff0-b6e8-0ee87bd5bdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:23.911625 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca3529-8ed0-4ff0-b6e8-0ee87bd5bdd1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:23.911899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca3529-8ed0-4ff0-b6e8-0ee87bd5bdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:23.912166 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca3529-8ed0-4ff0-b6e8-0ee87bd5bdd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:23.913248 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca3529-8ed0-4ff0-b6e8-0ee87bd5bdd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:23.914647 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca3529-8ed0-4ff0-b6e8-0ee87bd5bdd1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:23.916199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 425/859] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:13:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:23.922525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0527369-5241-4c96-90bc-afc68673a26c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:24.016804 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0527369-5241-4c96-90bc-afc68673a26c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:24.040983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:24.050391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d7d43-17a4-4359-9919-ae64e05b534c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:24.050983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d7d43-17a4-4359-9919-ae64e05b534c None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:24.051274 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d7d43-17a4-4359-9919-ae64e05b534c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:24.051543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d7d43-17a4-4359-9919-ae64e05b534c None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:24.052182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d7d43-17a4-4359-9919-ae64e05b534c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:24.053594 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d7d43-17a4-4359-9919-ae64e05b534c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:24.078317 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 435/860] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Oct 14 02:13:23 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:13:26.291764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f28559-3366-4e03-987c-4f8b9e7a4613 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:26.292292 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f28559-3366-4e03-987c-4f8b9e7a4613 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:26.292292 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f28559-3366-4e03-987c-4f8b9e7a4613 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:26.293668 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 426/861] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:26.298822 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c99e92-3ef2-443c-bed2-00fb7328ff22 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:26.299213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c99e92-3ef2-443c-bed2-00fb7328ff22 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:26.299394 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c99e92-3ef2-443c-bed2-00fb7328ff22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:26.317568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-40c99e92-3ef2-443c-bed2-00fb7328ff22 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:26.318727 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-40c99e92-3ef2-443c-bed2-00fb7328ff22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:26.383174 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 436/862] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:26.653183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f28559-3366-4e03-987c-4f8b9e7a4613 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:26.669212 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:26.754761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f28559-3366-4e03-987c-4f8b9e7a4613 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:26.787224 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774c434-ec87-49e4-9427-48de22d1c697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:26.787847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774c434-ec87-49e4-9427-48de22d1c697 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:26.788124 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774c434-ec87-49e4-9427-48de22d1c697 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:26.788396 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774c434-ec87-49e4-9427-48de22d1c697 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:26.789498 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774c434-ec87-49e4-9427-48de22d1c697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:26.790915 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774c434-ec87-49e4-9427-48de22d1c697 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:26.793000 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 427/863] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:13:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:26.798908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c99e92-3ef2-443c-bed2-00fb7328ff22 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:26.892632 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c99e92-3ef2-443c-bed2-00fb7328ff22 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:26.901292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:26.927360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a29bb2-9d71-47aa-b8b1-ae50c6d7d10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:26.927829 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a29bb2-9d71-47aa-b8b1-ae50c6d7d10a None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:26.928104 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a29bb2-9d71-47aa-b8b1-ae50c6d7d10a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:26.928373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a29bb2-9d71-47aa-b8b1-ae50c6d7d10a None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:26.929022 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a29bb2-9d71-47aa-b8b1-ae50c6d7d10a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:26.930443 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a29bb2-9d71-47aa-b8b1-ae50c6d7d10a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:26.949697 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 437/864] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:13:26 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:13:29.087005 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35045a-9da9-422d-9838-2694b28ecb9d None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:29.087555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35045a-9da9-422d-9838-2694b28ecb9d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:29.087555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35045a-9da9-422d-9838-2694b28ecb9d None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:29.088886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 428/865] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:29.094341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ab205-07d2-40ca-b2d6-2a3c6fe9680f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:29.094719 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ab205-07d2-40ca-b2d6-2a3c6fe9680f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:29.094885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ab205-07d2-40ca-b2d6-2a3c6fe9680f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:29.113116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-088ab205-07d2-40ca-b2d6-2a3c6fe9680f None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:29.114289 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-088ab205-07d2-40ca-b2d6-2a3c6fe9680f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:29.178865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 438/866] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:29.449192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35045a-9da9-422d-9838-2694b28ecb9d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:29.528481 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:29.547816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35045a-9da9-422d-9838-2694b28ecb9d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:29.580358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad909b6f-0776-4837-8c02-da0bccff423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:29.580979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad909b6f-0776-4837-8c02-da0bccff423a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:29.581259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad909b6f-0776-4837-8c02-da0bccff423a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:29.581532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad909b6f-0776-4837-8c02-da0bccff423a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:29.591322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad909b6f-0776-4837-8c02-da0bccff423a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-ad909b6f-0776-4837-8c02-da0bccff423a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:13:29.596080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:29.597150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:29.597984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:29.597984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:29.597984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:29.597984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:29.597984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:13:29.597984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:13:29.597984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:13:29.597984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:13:29.597984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:13:29.597984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:13:29.597984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:13:29.597984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:29.597984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 429/867] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:13:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:29.603119 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ab205-07d2-40ca-b2d6-2a3c6fe9680f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:29.692400 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:29.699857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ab205-07d2-40ca-b2d6-2a3c6fe9680f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:29.734619 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb3cb6-8fdc-488c-9bdb-f925d09f1b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:29.735274 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb3cb6-8fdc-488c-9bdb-f925d09f1b94 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:29.735554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb3cb6-8fdc-488c-9bdb-f925d09f1b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:29.735826 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb3cb6-8fdc-488c-9bdb-f925d09f1b94 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:29.736488 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb3cb6-8fdc-488c-9bdb-f925d09f1b94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:29.736787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb3cb6-8fdc-488c-9bdb-f925d09f1b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:29.738146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb3cb6-8fdc-488c-9bdb-f925d09f1b94 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:29.747018 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 439/868] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:13:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:29.752774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad909b6f-0776-4837-8c02-da0bccff423a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:29.848695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad909b6f-0776-4837-8c02-da0bccff423a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:29.882427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403dc32-0dab-4d60-832c-4efea9ac05dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:29.883039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403dc32-0dab-4d60-832c-4efea9ac05dc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:29.883351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403dc32-0dab-4d60-832c-4efea9ac05dc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:29.883622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403dc32-0dab-4d60-832c-4efea9ac05dc None admin] PATH_INFO: `/v3/users/180591b536094e5e8675cd843118704e` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:29.894162 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:29.896594 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403dc32-0dab-4d60-832c-4efea9ac05dc None admin] RBAC: Authorizing `identity:get_user(user_id=180591b536094e5e8675cd843118704e)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:29.898563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403dc32-0dab-4d60-832c-4efea9ac05dc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:29.900340 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 430/869] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:13:29 2022] GET /identity/v3/users/180591b536094e5e8675cd843118704e => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:29.906522 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb3cb6-8fdc-488c-9bdb-f925d09f1b94 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:30.002064 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb3cb6-8fdc-488c-9bdb-f925d09f1b94 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:30.034008 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db754a9-dc07-4ce0-ac91-baa6da5b66a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:30.034617 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db754a9-dc07-4ce0-ac91-baa6da5b66a7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:30.034894 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db754a9-dc07-4ce0-ac91-baa6da5b66a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:30.035192 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db754a9-dc07-4ce0-ac91-baa6da5b66a7 None admin] PATH_INFO: `/v3/projects/fa3f6385716247c1915cbcdb432de99b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:30.036276 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db754a9-dc07-4ce0-ac91-baa6da5b66a7 None admin] RBAC: Authorizing `identity:get_project(project_id=fa3f6385716247c1915cbcdb432de99b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:30.037683 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db754a9-dc07-4ce0-ac91-baa6da5b66a7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:30.039272 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 440/870] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:13:29 2022] GET /identity/v3/projects/fa3f6385716247c1915cbcdb432de99b => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:30.045792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403dc32-0dab-4d60-832c-4efea9ac05dc None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:30.140139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403dc32-0dab-4d60-832c-4efea9ac05dc None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:30.172953 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a04f1c-da6e-4619-9459-e5460ef64aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:30.173594 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a04f1c-da6e-4619-9459-e5460ef64aff None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:30.173869 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a04f1c-da6e-4619-9459-e5460ef64aff None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:30.174137 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a04f1c-da6e-4619-9459-e5460ef64aff None admin] PATH_INFO: `/v3/projects/fa3f6385716247c1915cbcdb432de99b/users/180591b536094e5e8675cd843118704e/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:30.176008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a04f1c-da6e-4619-9459-e5460ef64aff None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3f6385716247c1915cbcdb432de99b, user_id=180591b536094e5e8675cd843118704e, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:30.177406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a04f1c-da6e-4619-9459-e5460ef64aff None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:30.189847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 431/871] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:13:30 2022] PUT /identity/v3/projects/fa3f6385716247c1915cbcdb432de99b/users/180591b536094e5e8675cd843118704e/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:13:32.296535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838a2fd-1e53-4803-975c-bbe2f13004fe None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:32.297081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838a2fd-1e53-4803-975c-bbe2f13004fe None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:32.297081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838a2fd-1e53-4803-975c-bbe2f13004fe None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:32.298411 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 441/872] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:32.303841 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1196b-03c8-421c-81fe-d9d1aaa2595a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:32.304257 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1196b-03c8-421c-81fe-d9d1aaa2595a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:32.304389 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1196b-03c8-421c-81fe-d9d1aaa2595a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:32.322514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-2be1196b-03c8-421c-81fe-d9d1aaa2595a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:32.324337 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2be1196b-03c8-421c-81fe-d9d1aaa2595a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:32.377259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:32.389651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 432/873] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:32.664030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838a2fd-1e53-4803-975c-bbe2f13004fe None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:32.690150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:32.767718 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838a2fd-1e53-4803-975c-bbe2f13004fe None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:32.801124 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e537a-91be-4df0-bbc0-bbd3446cdf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:32.801739 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e537a-91be-4df0-bbc0-bbd3446cdf46 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:32.802019 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e537a-91be-4df0-bbc0-bbd3446cdf46 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:32.802290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e537a-91be-4df0-bbc0-bbd3446cdf46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:32.812438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e537a-91be-4df0-bbc0-bbd3446cdf46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-1c3e537a-91be-4df0-bbc0-bbd3446cdf46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:13:32.817183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:32.818231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:32.819090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:32.819090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:32.819090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:32.819090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:32.819090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:13:32.819090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:13:32.819090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:13:32.819090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:13:32.819090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:13:32.819090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:13:32.819090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:13:32.819090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:13:32.819090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 442/874] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:13:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:32.825298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1196b-03c8-421c-81fe-d9d1aaa2595a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:32.920852 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1196b-03c8-421c-81fe-d9d1aaa2595a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:32.952515 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122ed37-6262-42d2-a34a-902d9eabb944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:32.953112 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122ed37-6262-42d2-a34a-902d9eabb944 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:32.953390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122ed37-6262-42d2-a34a-902d9eabb944 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:32.953666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122ed37-6262-42d2-a34a-902d9eabb944 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:32.954330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122ed37-6262-42d2-a34a-902d9eabb944 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:32.954635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122ed37-6262-42d2-a34a-902d9eabb944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:32.956010 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122ed37-6262-42d2-a34a-902d9eabb944 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:32.963720 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 433/875] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:13:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:32.970099 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e537a-91be-4df0-bbc0-bbd3446cdf46 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:33.056699 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:33.072389 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e537a-91be-4df0-bbc0-bbd3446cdf46 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:33.104406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d32487-33d5-46c7-877b-3fea00fac707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:33.105618 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d32487-33d5-46c7-877b-3fea00fac707 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:33.105893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d32487-33d5-46c7-877b-3fea00fac707 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:33.106162 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d32487-33d5-46c7-877b-3fea00fac707 None admin] PATH_INFO: `/v3/users/180591b536094e5e8675cd843118704e` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:33.107284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d32487-33d5-46c7-877b-3fea00fac707 None admin] RBAC: Authorizing `identity:get_user(user_id=180591b536094e5e8675cd843118704e)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:33.108696 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d32487-33d5-46c7-877b-3fea00fac707 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:33.110233 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 443/876] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:13:32 2022] GET /identity/v3/users/180591b536094e5e8675cd843118704e => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:33.116043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122ed37-6262-42d2-a34a-902d9eabb944 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:33.168301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:33.213508 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122ed37-6262-42d2-a34a-902d9eabb944 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:33.246522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ccab4-aee4-4922-bc80-30a442e8b22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:33.247148 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ccab4-aee4-4922-bc80-30a442e8b22d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:33.247430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ccab4-aee4-4922-bc80-30a442e8b22d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:33.247704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ccab4-aee4-4922-bc80-30a442e8b22d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:33.312412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ccab4-aee4-4922-bc80-30a442e8b22d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:33.313842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ccab4-aee4-4922-bc80-30a442e8b22d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:33.362044 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:33.408204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 434/877] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:13:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:13:33.414087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d32487-33d5-46c7-877b-3fea00fac707 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:33.509663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d32487-33d5-46c7-877b-3fea00fac707 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:33.540910 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b61d2c-ab65-4fa1-9b86-d67801eacbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:33.541519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b61d2c-ab65-4fa1-9b86-d67801eacbbf None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:33.541792 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b61d2c-ab65-4fa1-9b86-d67801eacbbf None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:33.542709 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b61d2c-ab65-4fa1-9b86-d67801eacbbf None admin] PATH_INFO: `/v3/projects/fa3f6385716247c1915cbcdb432de99b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:33.543832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b61d2c-ab65-4fa1-9b86-d67801eacbbf None admin] RBAC: Authorizing `identity:get_project(project_id=fa3f6385716247c1915cbcdb432de99b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:33.545245 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b61d2c-ab65-4fa1-9b86-d67801eacbbf None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:33.546811 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 444/878] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:13:33 2022] GET /identity/v3/projects/fa3f6385716247c1915cbcdb432de99b => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:33.552721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ccab4-aee4-4922-bc80-30a442e8b22d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:33.646143 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ccab4-aee4-4922-bc80-30a442e8b22d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:33.678761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67472757-4f6b-4881-a2e4-b90213af174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:33.679377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67472757-4f6b-4881-a2e4-b90213af174c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:33.679651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67472757-4f6b-4881-a2e4-b90213af174c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:33.679915 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67472757-4f6b-4881-a2e4-b90213af174c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:33.680625 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67472757-4f6b-4881-a2e4-b90213af174c None admin] RBAC: Adding query filter params (role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b, scope.project.id=fa3f6385716247c1915cbcdb432de99b, user.id=180591b536094e5e8675cd843118704e) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:33.680927 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67472757-4f6b-4881-a2e4-b90213af174c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:33.682330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67472757-4f6b-4881-a2e4-b90213af174c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:33.687917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 435/879] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:13:33 2022] GET /identity/v3/role_assignments?user.id=180591b536094e5e8675cd843118704e&scope.project.id=fa3f6385716247c1915cbcdb432de99b&role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:35.812402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3010a-95c7-4422-b54f-aea3d859e9ea None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:35.812951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3010a-95c7-4422-b54f-aea3d859e9ea None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:35.812951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3010a-95c7-4422-b54f-aea3d859e9ea None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:35.814256 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 445/880] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:35.819605 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8e28-59ca-47ea-b7cf-40548420b270 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:35.820016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8e28-59ca-47ea-b7cf-40548420b270 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:35.820159 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8e28-59ca-47ea-b7cf-40548420b270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:35.838240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-908f8e28-59ca-47ea-b7cf-40548420b270 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:35.839701 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-908f8e28-59ca-47ea-b7cf-40548420b270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:35.850408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:35.906139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 436/881] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:36.180532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3010a-95c7-4422-b54f-aea3d859e9ea None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:36.203030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:36.279371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3010a-95c7-4422-b54f-aea3d859e9ea None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:36.311822 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52ca65-221a-4745-bc80-9eb32641afb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:36.312416 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52ca65-221a-4745-bc80-9eb32641afb0 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:36.312689 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52ca65-221a-4745-bc80-9eb32641afb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:36.312961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52ca65-221a-4745-bc80-9eb32641afb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:36.313617 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52ca65-221a-4745-bc80-9eb32641afb0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:36.314964 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52ca65-221a-4745-bc80-9eb32641afb0 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:36.329585 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 446/882] 10.222.0.18 () {60 vars in 1138 bytes} [Fri Oct 14 02:13:36 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:13:38.462101 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28744c6b-6151-4f17-ab00-be1fb4268175 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:38.462708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28744c6b-6151-4f17-ab00-be1fb4268175 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:38.462708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28744c6b-6151-4f17-ab00-be1fb4268175 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:38.464528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 437/883] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:38.469683 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34b03c-af5f-456e-bf14-a9f09facb785 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:38.470131 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34b03c-af5f-456e-bf14-a9f09facb785 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:38.470227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34b03c-af5f-456e-bf14-a9f09facb785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:38.488706 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5b34b03c-af5f-456e-bf14-a9f09facb785 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:38.489888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5b34b03c-af5f-456e-bf14-a9f09facb785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:38.555686 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 447/884] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:38.829857 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28744c6b-6151-4f17-ab00-be1fb4268175 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:38.929435 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28744c6b-6151-4f17-ab00-be1fb4268175 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:38.961929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c90496-1b73-4d91-b3e6-8235f8dd60f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:38.962546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c90496-1b73-4d91-b3e6-8235f8dd60f0 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:38.962826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c90496-1b73-4d91-b3e6-8235f8dd60f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:38.963122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c90496-1b73-4d91-b3e6-8235f8dd60f0 None admin] PATH_INFO: `/v3/domains/6f56899e58d24354ad75d1486dcde114` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:38.974029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c90496-1b73-4d91-b3e6-8235f8dd60f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6f56899e58d24354ad75d1486dcde114)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:38.975486 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c90496-1b73-4d91-b3e6-8235f8dd60f0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:38.977027 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 438/885] 10.222.0.18 () {56 vars in 1183 bytes} [Fri Oct 14 02:13:38 2022] GET /identity/v3/domains/6f56899e58d24354ad75d1486dcde114 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:38.983373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34b03c-af5f-456e-bf14-a9f09facb785 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:38.994516 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:39.080048 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34b03c-af5f-456e-bf14-a9f09facb785 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:39.114901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4419097-fd3e-4412-b1c6-1f82e3188754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:39.115535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4419097-fd3e-4412-b1c6-1f82e3188754 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:39.115815 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4419097-fd3e-4412-b1c6-1f82e3188754 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:39.116089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4419097-fd3e-4412-b1c6-1f82e3188754 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:39.116732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4419097-fd3e-4412-b1c6-1f82e3188754 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:39.118160 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4419097-fd3e-4412-b1c6-1f82e3188754 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:39.143917 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 448/886] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Oct 14 02:13:38 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:13:41.297507 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfa319-164e-44ac-82b3-950d68a32c99 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:41.298056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfa319-164e-44ac-82b3-950d68a32c99 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:41.298056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfa319-164e-44ac-82b3-950d68a32c99 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:41.299408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 439/887] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:41.304729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f473e8c-0d56-4d4d-a096-b923865b5799 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:41.305112 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f473e8c-0d56-4d4d-a096-b923865b5799 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:41.305284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f473e8c-0d56-4d4d-a096-b923865b5799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:41.325050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8f473e8c-0d56-4d4d-a096-b923865b5799 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:41.326212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8f473e8c-0d56-4d4d-a096-b923865b5799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:41.379669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:41.391527 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 449/888] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:41.661300 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfa319-164e-44ac-82b3-950d68a32c99 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:41.670929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:41.762163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfa319-164e-44ac-82b3-950d68a32c99 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:41.794224 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5969b-b58b-47e3-9dd4-7a74a72580e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:41.794850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5969b-b58b-47e3-9dd4-7a74a72580e1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:41.795159 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5969b-b58b-47e3-9dd4-7a74a72580e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:41.795437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5969b-b58b-47e3-9dd4-7a74a72580e1 None admin] PATH_INFO: `/v3/domains/6f56899e58d24354ad75d1486dcde114` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:41.796546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5969b-b58b-47e3-9dd4-7a74a72580e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6f56899e58d24354ad75d1486dcde114)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:41.797968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5969b-b58b-47e3-9dd4-7a74a72580e1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:41.799556 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 440/889] 10.222.0.18 () {56 vars in 1183 bytes} [Fri Oct 14 02:13:41 2022] GET /identity/v3/domains/6f56899e58d24354ad75d1486dcde114 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:13:41.805259 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f473e8c-0d56-4d4d-a096-b923865b5799 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:41.899630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f473e8c-0d56-4d4d-a096-b923865b5799 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:41.932833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52668f84-6767-47db-be15-240a351b3097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:41.933367 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52668f84-6767-47db-be15-240a351b3097 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:41.933641 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52668f84-6767-47db-be15-240a351b3097 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:41.933906 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52668f84-6767-47db-be15-240a351b3097 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:41.934540 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52668f84-6767-47db-be15-240a351b3097 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:41.936050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52668f84-6767-47db-be15-240a351b3097 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:41.956088 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 450/890] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:13:41 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:13:44.101738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419e151-45b8-4d4d-801b-db7cd558dc38 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:44.102325 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419e151-45b8-4d4d-801b-db7cd558dc38 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:44.102325 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419e151-45b8-4d4d-801b-db7cd558dc38 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:44.104126 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 441/891] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:44.109198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e1067-9e19-47fe-b2b3-4e5bd7d9774d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:44.109604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e1067-9e19-47fe-b2b3-4e5bd7d9774d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:44.109748 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e1067-9e19-47fe-b2b3-4e5bd7d9774d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:44.128335 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-429e1067-9e19-47fe-b2b3-4e5bd7d9774d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:44.129515 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-429e1067-9e19-47fe-b2b3-4e5bd7d9774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:44.172063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:44.194621 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 451/892] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:44.469842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419e151-45b8-4d4d-801b-db7cd558dc38 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:44.543780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:44.568878 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419e151-45b8-4d4d-801b-db7cd558dc38 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:44.599763 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd70152-743f-4227-9874-67266dd41b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:44.600380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd70152-743f-4227-9874-67266dd41b81 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:44.600655 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd70152-743f-4227-9874-67266dd41b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:44.600920 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd70152-743f-4227-9874-67266dd41b81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:44.611546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd70152-743f-4227-9874-67266dd41b81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-fbd70152-743f-4227-9874-67266dd41b81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:13:44.616216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:44.617322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:44.618358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:44.618358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:44.618358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:44.618358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:44.618358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:13:44.618358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:13:44.618358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:13:44.618358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:13:44.618358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:13:44.618358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:13:44.618358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:13:44.618358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:13:44.618358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 442/893] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:13:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:44.624017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e1067-9e19-47fe-b2b3-4e5bd7d9774d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:44.718774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e1067-9e19-47fe-b2b3-4e5bd7d9774d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:44.752630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b4816-4ecf-4875-9f13-5a09a504b753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:44.753236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b4816-4ecf-4875-9f13-5a09a504b753 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:44.753516 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b4816-4ecf-4875-9f13-5a09a504b753 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:44.753787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b4816-4ecf-4875-9f13-5a09a504b753 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:44.754444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b4816-4ecf-4875-9f13-5a09a504b753 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:44.754744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b4816-4ecf-4875-9f13-5a09a504b753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:44.756147 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b4816-4ecf-4875-9f13-5a09a504b753 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:44.764783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 452/894] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:13:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:44.770440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd70152-743f-4227-9874-67266dd41b81 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:44.870864 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd70152-743f-4227-9874-67266dd41b81 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:44.902454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc37b87-39b7-4a3a-8fe9-1234cd164994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:44.903094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc37b87-39b7-4a3a-8fe9-1234cd164994 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:44.903371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc37b87-39b7-4a3a-8fe9-1234cd164994 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:44.903641 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc37b87-39b7-4a3a-8fe9-1234cd164994 None admin] PATH_INFO: `/v3/users/57efbabf68b84466b353c36098ecd47a` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:44.905429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:44.917330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc37b87-39b7-4a3a-8fe9-1234cd164994 None admin] RBAC: Authorizing `identity:get_user(user_id=57efbabf68b84466b353c36098ecd47a)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:44.918762 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc37b87-39b7-4a3a-8fe9-1234cd164994 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:44.920457 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 443/895] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:13:44 2022] GET /identity/v3/users/57efbabf68b84466b353c36098ecd47a => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:44.926216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b4816-4ecf-4875-9f13-5a09a504b753 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:44.966917 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:45.023732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b4816-4ecf-4875-9f13-5a09a504b753 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:45.058008 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55281dbe-22ff-487c-8eb5-ad38699c9433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:45.058630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55281dbe-22ff-487c-8eb5-ad38699c9433 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:45.058909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55281dbe-22ff-487c-8eb5-ad38699c9433 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:45.059212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55281dbe-22ff-487c-8eb5-ad38699c9433 None admin] PATH_INFO: `/v3/projects/6f7768d69c594d3aa48433190fe7bd53` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:45.060319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55281dbe-22ff-487c-8eb5-ad38699c9433 None admin] RBAC: Authorizing `identity:get_project(project_id=6f7768d69c594d3aa48433190fe7bd53)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:45.061745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55281dbe-22ff-487c-8eb5-ad38699c9433 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:45.063383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 453/896] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:13:44 2022] GET /identity/v3/projects/6f7768d69c594d3aa48433190fe7bd53 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:45.069264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc37b87-39b7-4a3a-8fe9-1234cd164994 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:45.165930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc37b87-39b7-4a3a-8fe9-1234cd164994 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:45.199328 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053bfef-1c4a-40d3-8117-8fbe39772a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:45.199984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053bfef-1c4a-40d3-8117-8fbe39772a55 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:45.200265 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053bfef-1c4a-40d3-8117-8fbe39772a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:45.200534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053bfef-1c4a-40d3-8117-8fbe39772a55 None admin] PATH_INFO: `/v3/projects/6f7768d69c594d3aa48433190fe7bd53/users/57efbabf68b84466b353c36098ecd47a/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:45.202410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053bfef-1c4a-40d3-8117-8fbe39772a55 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f7768d69c594d3aa48433190fe7bd53, user_id=57efbabf68b84466b353c36098ecd47a, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:45.203853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053bfef-1c4a-40d3-8117-8fbe39772a55 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:45.216385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 444/897] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:13:45 2022] PUT /identity/v3/projects/6f7768d69c594d3aa48433190fe7bd53/users/57efbabf68b84466b353c36098ecd47a/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:13:47.313113 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfcfb8-a0bc-45c2-9145-e2cc5ebd2d10 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:47.313659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfcfb8-a0bc-45c2-9145-e2cc5ebd2d10 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:47.313659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfcfb8-a0bc-45c2-9145-e2cc5ebd2d10 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:47.314995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 454/898] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:13:47.320265 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff86dcd-1d00-4612-93b8-39084be25d1b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:47.320700 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff86dcd-1d00-4612-93b8-39084be25d1b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:47.320795 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff86dcd-1d00-4612-93b8-39084be25d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:47.339020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-bff86dcd-1d00-4612-93b8-39084be25d1b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:13:47.340213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bff86dcd-1d00-4612-93b8-39084be25d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:13:47.384015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:47.404220 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 445/899] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:13:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:13:47.674604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfcfb8-a0bc-45c2-9145-e2cc5ebd2d10 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:47.774940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfcfb8-a0bc-45c2-9145-e2cc5ebd2d10 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:47.784423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:47.809789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51dd7c-d79d-4a8b-9532-1ac79d0378f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:47.810402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51dd7c-d79d-4a8b-9532-1ac79d0378f4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:47.810675 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51dd7c-d79d-4a8b-9532-1ac79d0378f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:47.810940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51dd7c-d79d-4a8b-9532-1ac79d0378f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:47.820549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51dd7c-d79d-4a8b-9532-1ac79d0378f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-4b51dd7c-d79d-4a8b-9532-1ac79d0378f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:13:47.825324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:13:47.826410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:13:47.827477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:13:47.827477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:13:47.827477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:13:47.827477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:13:47.827477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:13:47.827477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:13:47.827477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:13:47.827477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:13:47.827477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:13:47.827477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:13:47.827477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:13:47.827477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:13:47.827477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 455/900] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:13:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:13:47.832718 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff86dcd-1d00-4612-93b8-39084be25d1b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:47.927203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff86dcd-1d00-4612-93b8-39084be25d1b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:47.958837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f1e3f-7d80-48d8-b2fe-6bc700ff3430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:47.959468 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f1e3f-7d80-48d8-b2fe-6bc700ff3430 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:47.959744 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f1e3f-7d80-48d8-b2fe-6bc700ff3430 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:47.960016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f1e3f-7d80-48d8-b2fe-6bc700ff3430 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:47.960667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f1e3f-7d80-48d8-b2fe-6bc700ff3430 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:47.960966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f1e3f-7d80-48d8-b2fe-6bc700ff3430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:47.962316 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f1e3f-7d80-48d8-b2fe-6bc700ff3430 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:47.970453 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 446/901] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:13:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:47.976191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51dd7c-d79d-4a8b-9532-1ac79d0378f4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:48.071578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51dd7c-d79d-4a8b-9532-1ac79d0378f4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:48.104390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05062e8-0162-43c2-8c9d-165b03e2a7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:48.104995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05062e8-0162-43c2-8c9d-165b03e2a7fd None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:48.105263 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05062e8-0162-43c2-8c9d-165b03e2a7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:48.105527 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05062e8-0162-43c2-8c9d-165b03e2a7fd None admin] PATH_INFO: `/v3/users/57efbabf68b84466b353c36098ecd47a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:48.106619 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05062e8-0162-43c2-8c9d-165b03e2a7fd None admin] RBAC: Authorizing `identity:get_user(user_id=57efbabf68b84466b353c36098ecd47a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:48.108056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05062e8-0162-43c2-8c9d-165b03e2a7fd None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:48.109576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 456/902] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Oct 14 02:13:47 2022] GET /identity/v3/users/57efbabf68b84466b353c36098ecd47a => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:48.115218 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f1e3f-7d80-48d8-b2fe-6bc700ff3430 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:48.159460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:48.211256 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f1e3f-7d80-48d8-b2fe-6bc700ff3430 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:48.243657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b221109-713a-444a-bb72-faffb4045abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:48.244375 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b221109-713a-444a-bb72-faffb4045abf None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:48.244648 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b221109-713a-444a-bb72-faffb4045abf None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:48.245169 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b221109-713a-444a-bb72-faffb4045abf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:48.308421 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b221109-713a-444a-bb72-faffb4045abf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:48.309842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b221109-713a-444a-bb72-faffb4045abf None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:48.351727 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:48.404591 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 447/903] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:13:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:13:48.410435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05062e8-0162-43c2-8c9d-165b03e2a7fd None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:48.444893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:48.508643 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05062e8-0162-43c2-8c9d-165b03e2a7fd None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:48.541459 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a776e-7e1a-43e8-b451-169aa8502a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:48.542349 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a776e-7e1a-43e8-b451-169aa8502a49 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:48.542624 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a776e-7e1a-43e8-b451-169aa8502a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:48.542892 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a776e-7e1a-43e8-b451-169aa8502a49 None admin] PATH_INFO: `/v3/projects/6f7768d69c594d3aa48433190fe7bd53` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:48.544027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a776e-7e1a-43e8-b451-169aa8502a49 None admin] RBAC: Authorizing `identity:get_project(project_id=6f7768d69c594d3aa48433190fe7bd53)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:48.545444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a776e-7e1a-43e8-b451-169aa8502a49 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:48.547040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 457/904] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Oct 14 02:13:48 2022] GET /identity/v3/projects/6f7768d69c594d3aa48433190fe7bd53 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:48.553291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b221109-713a-444a-bb72-faffb4045abf None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:13:48.647408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b221109-713a-444a-bb72-faffb4045abf None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:13:48.679942 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc101f7-10ee-47ef-8241-df980ac94d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:13:48.680534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc101f7-10ee-47ef-8241-df980ac94d4d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:13:48.680813 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc101f7-10ee-47ef-8241-df980ac94d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:13:48.681084 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc101f7-10ee-47ef-8241-df980ac94d4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:13:48.682328 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc101f7-10ee-47ef-8241-df980ac94d4d None admin] RBAC: Adding query filter params (role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b, scope.project.id=6f7768d69c594d3aa48433190fe7bd53, user.id=57efbabf68b84466b353c36098ecd47a) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:13:48.682629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc101f7-10ee-47ef-8241-df980ac94d4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:13:48.684056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc101f7-10ee-47ef-8241-df980ac94d4d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:13:48.689810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 448/905] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:13:48 2022] GET /identity/v3/role_assignments?user.id=57efbabf68b84466b353c36098ecd47a&scope.project.id=6f7768d69c594d3aa48433190fe7bd53&role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:13:58.536870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=48 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:13:58.687032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:44.143971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53829722-8180-4cc0-acc6-724721d74b6c None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:44.144566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53829722-8180-4cc0-acc6-724721d74b6c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:44.144566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53829722-8180-4cc0-acc6-724721d74b6c None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:44.145924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 458/906] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:14:44.152674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaef525-d86d-4879-afa4-66e21cfa1fed None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:44.153210 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaef525-d86d-4879-afa4-66e21cfa1fed None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:44.153318 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaef525-d86d-4879-afa4-66e21cfa1fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:44.174411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-eaaef525-d86d-4879-afa4-66e21cfa1fed None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:14:44.175768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-eaaef525-d86d-4879-afa4-66e21cfa1fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:14:44.262939 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 449/907] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:14:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:14:44.543401 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53829722-8180-4cc0-acc6-724721d74b6c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:44.664333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53829722-8180-4cc0-acc6-724721d74b6c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:44.702066 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c522f8c-b921-4f27-925b-54eb8ba9c0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:44.702579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c522f8c-b921-4f27-925b-54eb8ba9c0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:44.702859 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c522f8c-b921-4f27-925b-54eb8ba9c0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:44.703173 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c522f8c-b921-4f27-925b-54eb8ba9c0b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:44.714460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c522f8c-b921-4f27-925b-54eb8ba9c0b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:44.715967 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c522f8c-b921-4f27-925b-54eb8ba9c0b3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-1c522f8c-b921-4f27-925b-54eb8ba9c0b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:14:44.732455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:44.733808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:14:44.735692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:14:44.735692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:14:44.735692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:14:44.735692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 459/908] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:14:44.739894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaef525-d86d-4879-afa4-66e21cfa1fed None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:44.864755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaef525-d86d-4879-afa4-66e21cfa1fed None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:44.897987 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:44.905549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269e0e0-4769-47d0-9a02-06e29986ad43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:44.905917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269e0e0-4769-47d0-9a02-06e29986ad43 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:44.906207 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269e0e0-4769-47d0-9a02-06e29986ad43 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:44.906471 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269e0e0-4769-47d0-9a02-06e29986ad43 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:44.907203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269e0e0-4769-47d0-9a02-06e29986ad43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:44.907452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269e0e0-4769-47d0-9a02-06e29986ad43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:44.908849 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269e0e0-4769-47d0-9a02-06e29986ad43 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:44.922415 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 450/909] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:44.927966 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c522f8c-b921-4f27-925b-54eb8ba9c0b3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:44.991509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:45.060046 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c522f8c-b921-4f27-925b-54eb8ba9c0b3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:45.106532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780953fc-2f19-4441-86bd-ae5fe78804a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:45.107238 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780953fc-2f19-4441-86bd-ae5fe78804a3 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:45.107540 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780953fc-2f19-4441-86bd-ae5fe78804a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:45.107855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780953fc-2f19-4441-86bd-ae5fe78804a3 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:45.108447 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780953fc-2f19-4441-86bd-ae5fe78804a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:45.109903 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780953fc-2f19-4441-86bd-ae5fe78804a3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:45.133879 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 460/910] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Oct 14 02:14:44 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:14:47.462922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68e6a5-ba54-476a-9e24-60bd18fd6441 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:47.463510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68e6a5-ba54-476a-9e24-60bd18fd6441 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:47.463510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68e6a5-ba54-476a-9e24-60bd18fd6441 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:47.467099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 451/911] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:14:47.470730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bd895-5d1e-4fe7-8863-e1803558d3a9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:47.471266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bd895-5d1e-4fe7-8863-e1803558d3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:47.471354 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bd895-5d1e-4fe7-8863-e1803558d3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:47.493603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-077bd895-5d1e-4fe7-8863-e1803558d3a9 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:14:47.494785 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-077bd895-5d1e-4fe7-8863-e1803558d3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:14:47.579905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 461/912] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:14:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:14:47.877211 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68e6a5-ba54-476a-9e24-60bd18fd6441 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:47.999896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68e6a5-ba54-476a-9e24-60bd18fd6441 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:48.039121 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654bfb5-5637-4b7d-b85b-eddfb1eee4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:48.039780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654bfb5-5637-4b7d-b85b-eddfb1eee4ec None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:48.040118 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654bfb5-5637-4b7d-b85b-eddfb1eee4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:48.040369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654bfb5-5637-4b7d-b85b-eddfb1eee4ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:48.052467 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654bfb5-5637-4b7d-b85b-eddfb1eee4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-5654bfb5-5637-4b7d-b85b-eddfb1eee4ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:14:48.057442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:48.061643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:48.062483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:48.062483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:48.062483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:48.062483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:48.062483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:14:48.062483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:14:48.062483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:14:48.062483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:14:48.062483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:14:48.062483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:14:48.062483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:14:48.062483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:14:48.062483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 452/913] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:14:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:14:48.065192 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bd895-5d1e-4fe7-8863-e1803558d3a9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:48.113442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:48.188548 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bd895-5d1e-4fe7-8863-e1803558d3a9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:48.229058 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a9b3e-c48f-4237-b2b5-05f813c2fb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:48.229852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a9b3e-c48f-4237-b2b5-05f813c2fb98 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:48.230170 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a9b3e-c48f-4237-b2b5-05f813c2fb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:48.230463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a9b3e-c48f-4237-b2b5-05f813c2fb98 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:48.231117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a9b3e-c48f-4237-b2b5-05f813c2fb98 None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:48.231511 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a9b3e-c48f-4237-b2b5-05f813c2fb98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:48.233041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a9b3e-c48f-4237-b2b5-05f813c2fb98 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:48.242744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 462/914] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Oct 14 02:14:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:48.248362 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654bfb5-5637-4b7d-b85b-eddfb1eee4ec None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:48.312313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:48.374367 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654bfb5-5637-4b7d-b85b-eddfb1eee4ec None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:48.413147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2698e936-5470-437d-9c57-8e87c9efbd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:48.414135 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698e936-5470-437d-9c57-8e87c9efbd82 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:48.414135 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698e936-5470-437d-9c57-8e87c9efbd82 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:48.414452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698e936-5470-437d-9c57-8e87c9efbd82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:48.426444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2698e936-5470-437d-9c57-8e87c9efbd82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:48.428793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2698e936-5470-437d-9c57-8e87c9efbd82 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-2698e936-5470-437d-9c57-8e87c9efbd82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:14:48.452470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:48.460514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:14:48.461448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:14:48.461448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:14:48.461448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:14:48.461448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 453/915] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:14:48.461682 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a9b3e-c48f-4237-b2b5-05f813c2fb98 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:48.576689 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:48.581104 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a9b3e-c48f-4237-b2b5-05f813c2fb98 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:48.626480 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b82136-c977-4d88-9562-a103a4a9e4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:48.627203 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b82136-c977-4d88-9562-a103a4a9e4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:48.627410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b82136-c977-4d88-9562-a103a4a9e4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:48.627642 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b82136-c977-4d88-9562-a103a4a9e4d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:48.628281 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b82136-c977-4d88-9562-a103a4a9e4d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:48.628585 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b82136-c977-4d88-9562-a103a4a9e4d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:48.629946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b82136-c977-4d88-9562-a103a4a9e4d3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:48.642490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 463/916] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:14:48.648904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2698e936-5470-437d-9c57-8e87c9efbd82 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:48.766994 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2698e936-5470-437d-9c57-8e87c9efbd82 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:48.768678 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:48.811985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a4df3-5410-4b24-be28-b3106fd5a128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:48.812994 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a4df3-5410-4b24-be28-b3106fd5a128 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:48.813322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a4df3-5410-4b24-be28-b3106fd5a128 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:48.813594 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a4df3-5410-4b24-be28-b3106fd5a128 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:48.823669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a4df3-5410-4b24-be28-b3106fd5a128 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:48.825425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a4df3-5410-4b24-be28-b3106fd5a128 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-f57a4df3-5410-4b24-be28-b3106fd5a128 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:14:48.839132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:48.840161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:48.841088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:48.841088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:48.841088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:14:48.841088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:14:48.841088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:14:48.841088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:14:48.841088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:14:48.841088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:14:48.841088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:14:48.841088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:14:48.841088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 454/917] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:14:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:14:48.845953 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b82136-c977-4d88-9562-a103a4a9e4d3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:48.962443 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b82136-c977-4d88-9562-a103a4a9e4d3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:48.999847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abad13-9176-45af-b452-9f02f5133fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:49.000492 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abad13-9176-45af-b452-9f02f5133fad None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:49.000772 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abad13-9176-45af-b452-9f02f5133fad None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:49.001042 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abad13-9176-45af-b452-9f02f5133fad None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:49.001770 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abad13-9176-45af-b452-9f02f5133fad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:49.002078 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abad13-9176-45af-b452-9f02f5133fad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:49.003561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abad13-9176-45af-b452-9f02f5133fad None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:49.023990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 464/918] 10.222.0.18 () {56 vars in 1143 bytes} [Fri Oct 14 02:14:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:49.029986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a4df3-5410-4b24-be28-b3106fd5a128 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:49.142959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a4df3-5410-4b24-be28-b3106fd5a128 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:49.183920 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0338b-e6ea-40c1-82f7-b18133c8fac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:49.184561 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0338b-e6ea-40c1-82f7-b18133c8fac0 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:49.184849 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0338b-e6ea-40c1-82f7-b18133c8fac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:49.185115 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0338b-e6ea-40c1-82f7-b18133c8fac0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:49.196316 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0338b-e6ea-40c1-82f7-b18133c8fac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:49.197796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0338b-e6ea-40c1-82f7-b18133c8fac0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d4e0338b-e6ea-40c1-82f7-b18133c8fac0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:14:49.212998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:49.213984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:14:49.215488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:14:49.215488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:14:49.215488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:14:49.215488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 455/919] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:14:49.220071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abad13-9176-45af-b452-9f02f5133fad None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:49.239291 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:49.325642 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abad13-9176-45af-b452-9f02f5133fad None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:49.362160 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e65362-696e-4913-b027-55ab20705481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:49.362926 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e65362-696e-4913-b027-55ab20705481 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:49.363330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e65362-696e-4913-b027-55ab20705481 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:49.363702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e65362-696e-4913-b027-55ab20705481 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:49.364542 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e65362-696e-4913-b027-55ab20705481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:49.365569 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e65362-696e-4913-b027-55ab20705481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:49.367033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e65362-696e-4913-b027-55ab20705481 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:49.383921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 465/920] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:49.385765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0338b-e6ea-40c1-82f7-b18133c8fac0 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:49.403016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:49.513340 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0338b-e6ea-40c1-82f7-b18133c8fac0 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:49.553208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce0107-556e-4669-b51c-507c321617e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:49.553909 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce0107-556e-4669-b51c-507c321617e6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:49.554204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce0107-556e-4669-b51c-507c321617e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:49.554577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce0107-556e-4669-b51c-507c321617e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:49.562063 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce0107-556e-4669-b51c-507c321617e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:49.563700 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce0107-556e-4669-b51c-507c321617e6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-49ce0107-556e-4669-b51c-507c321617e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:14:49.576149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:49.577354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:49.578429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:14:49.578429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:14:49.578429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:14:49.578429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:14:49.578429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:14:49.578429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:14:49.578429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:14:49.578429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:14:49.582098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 456/921] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:14:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:14:49.583698 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e65362-696e-4913-b027-55ab20705481 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:49.670915 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:49.699361 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e65362-696e-4913-b027-55ab20705481 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:49.736575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965663f7-dc17-491c-ba55-1b43b1053723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:49.737494 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965663f7-dc17-491c-ba55-1b43b1053723 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:49.737990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965663f7-dc17-491c-ba55-1b43b1053723 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:49.738456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965663f7-dc17-491c-ba55-1b43b1053723 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:49.739456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965663f7-dc17-491c-ba55-1b43b1053723 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:49.739825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965663f7-dc17-491c-ba55-1b43b1053723 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:49.741551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965663f7-dc17-491c-ba55-1b43b1053723 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:49.756956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 466/922] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:14:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:49.764207 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce0107-556e-4669-b51c-507c321617e6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:49.858922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:49.883422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce0107-556e-4669-b51c-507c321617e6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:49.919142 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0fac6-663d-4e77-b8f1-2dd449dc8bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:49.919822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0fac6-663d-4e77-b8f1-2dd449dc8bdc None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:49.920103 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0fac6-663d-4e77-b8f1-2dd449dc8bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:49.920381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0fac6-663d-4e77-b8f1-2dd449dc8bdc None admin] PATH_INFO: `/v3/projects/f28d337589024cc88184355bd0ee8572/users/5e935ba63cdd4256874b99a2eb0a1a2c/roles/8adf2afafb0446a6992e9aeb380f645d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:49.935616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0fac6-663d-4e77-b8f1-2dd449dc8bdc None admin] RBAC: Authorizing `identity:create_grant(project_id=f28d337589024cc88184355bd0ee8572, user_id=5e935ba63cdd4256874b99a2eb0a1a2c, role_id=8adf2afafb0446a6992e9aeb380f645d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:49.937007 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0fac6-663d-4e77-b8f1-2dd449dc8bdc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:49.952022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 457/923] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:14:49 2022] PUT /identity/v3/projects/f28d337589024cc88184355bd0ee8572/users/5e935ba63cdd4256874b99a2eb0a1a2c/roles/8adf2afafb0446a6992e9aeb380f645d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:14:52.276687 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614e24e-8a0c-4943-9bc6-2aec5b94dcb5 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:52.277251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614e24e-8a0c-4943-9bc6-2aec5b94dcb5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:52.277341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614e24e-8a0c-4943-9bc6-2aec5b94dcb5 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:52.278785 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 467/924] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:14:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:14:52.284166 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e9cf4-dc60-420e-9e43-215fb8ead854 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:52.284593 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e9cf4-dc60-420e-9e43-215fb8ead854 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:52.284733 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e9cf4-dc60-420e-9e43-215fb8ead854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:52.303344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-de3e9cf4-dc60-420e-9e43-215fb8ead854 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:14:52.304872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-de3e9cf4-dc60-420e-9e43-215fb8ead854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:14:52.386564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 458/925] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:14:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:14:52.661333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614e24e-8a0c-4943-9bc6-2aec5b94dcb5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:52.777388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614e24e-8a0c-4943-9bc6-2aec5b94dcb5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:52.815618 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b2374-d358-4934-a171-fb1646276c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:52.816387 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b2374-d358-4934-a171-fb1646276c6f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:52.816733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b2374-d358-4934-a171-fb1646276c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:52.817004 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b2374-d358-4934-a171-fb1646276c6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:52.824252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:52.828480 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b2374-d358-4934-a171-fb1646276c6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-555b2374-d358-4934-a171-fb1646276c6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:14:52.834050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:52.835292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:52.836521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:52.836521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:52.836521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:52.836521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:52.836521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:14:52.836521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:14:52.836521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:14:52.836521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:14:52.836521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:14:52.836521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:14:52.836521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:14:52.836521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:14:52.836521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 468/926] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Oct 14 02:14:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:14:52.841414 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e9cf4-dc60-420e-9e43-215fb8ead854 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:52.901579 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:52.952455 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e9cf4-dc60-420e-9e43-215fb8ead854 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:52.991212 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d98e36-68b0-43a9-bc5f-bf79b8623968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:52.991977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d98e36-68b0-43a9-bc5f-bf79b8623968 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:52.992732 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d98e36-68b0-43a9-bc5f-bf79b8623968 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:52.993029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d98e36-68b0-43a9-bc5f-bf79b8623968 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:52.993792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d98e36-68b0-43a9-bc5f-bf79b8623968 None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:52.994258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d98e36-68b0-43a9-bc5f-bf79b8623968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:52.995706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d98e36-68b0-43a9-bc5f-bf79b8623968 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:53.013397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 459/927] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Oct 14 02:14:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:53.014601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b2374-d358-4934-a171-fb1646276c6f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:53.129188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b2374-d358-4934-a171-fb1646276c6f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:53.165541 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bb141-abb6-4e85-9bc6-819c5de8c165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:53.166364 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bb141-abb6-4e85-9bc6-819c5de8c165 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:53.166682 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bb141-abb6-4e85-9bc6-819c5de8c165 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:53.167158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bb141-abb6-4e85-9bc6-819c5de8c165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:53.182271 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bb141-abb6-4e85-9bc6-819c5de8c165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:53.183781 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bb141-abb6-4e85-9bc6-819c5de8c165 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-762bb141-abb6-4e85-9bc6-819c5de8c165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:14:53.199694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:53.201157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:14:53.202236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:14:53.202236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:14:53.202236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:14:53.202236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 469/928] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:14:53.206803 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d98e36-68b0-43a9-bc5f-bf79b8623968 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:53.312656 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d98e36-68b0-43a9-bc5f-bf79b8623968 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:53.329552 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:53.350894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cc72c-0834-4255-9798-3d65158efb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:53.351541 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cc72c-0834-4255-9798-3d65158efb1a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:53.351998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cc72c-0834-4255-9798-3d65158efb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:53.352127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cc72c-0834-4255-9798-3d65158efb1a None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:53.352727 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9cc72c-0834-4255-9798-3d65158efb1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:53.353025 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9cc72c-0834-4255-9798-3d65158efb1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:53.354382 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9cc72c-0834-4255-9798-3d65158efb1a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:53.367279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 460/929] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:53.375017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bb141-abb6-4e85-9bc6-819c5de8c165 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:53.437340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:53.499316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bb141-abb6-4e85-9bc6-819c5de8c165 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:53.537590 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536dc60d-b57e-486a-baee-c2987ba32cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:53.538227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc60d-b57e-486a-baee-c2987ba32cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:53.538508 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc60d-b57e-486a-baee-c2987ba32cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:53.540112 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc60d-b57e-486a-baee-c2987ba32cf6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:53.546892 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536dc60d-b57e-486a-baee-c2987ba32cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:53.548356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536dc60d-b57e-486a-baee-c2987ba32cf6 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-536dc60d-b57e-486a-baee-c2987ba32cf6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:14:53.560657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:53.561816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:53.562837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:53.562837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:53.562837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:14:53.562837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:14:53.562837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:14:53.562837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:14:53.562837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:14:53.562837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:14:53.562837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:14:53.562837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:14:53.562837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 470/930] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:14:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:14:53.568416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cc72c-0834-4255-9798-3d65158efb1a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:53.678315 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cc72c-0834-4255-9798-3d65158efb1a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:53.714788 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3dffc8-ecc1-47d4-9014-df2034460ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:53.715429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dffc8-ecc1-47d4-9014-df2034460ace None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:53.715748 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dffc8-ecc1-47d4-9014-df2034460ace None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:53.716067 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dffc8-ecc1-47d4-9014-df2034460ace None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:53.716799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3dffc8-ecc1-47d4-9014-df2034460ace None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:53.717160 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3dffc8-ecc1-47d4-9014-df2034460ace None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:53.718660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3dffc8-ecc1-47d4-9014-df2034460ace None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:53.738829 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 461/931] 10.222.0.18 () {56 vars in 1143 bytes} [Fri Oct 14 02:14:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:14:53.746170 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536dc60d-b57e-486a-baee-c2987ba32cf6 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:53.857091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536dc60d-b57e-486a-baee-c2987ba32cf6 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:53.868408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:53.894226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf5574-3fed-4cfd-9200-51a58da6aa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:53.894575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf5574-3fed-4cfd-9200-51a58da6aa5e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:53.894836 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf5574-3fed-4cfd-9200-51a58da6aa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:53.895201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf5574-3fed-4cfd-9200-51a58da6aa5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:53.969416 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf5574-3fed-4cfd-9200-51a58da6aa5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:53.971238 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf5574-3fed-4cfd-9200-51a58da6aa5e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:54.076546 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 471/932] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:14:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 14 02:14:54.082544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3dffc8-ecc1-47d4-9014-df2034460ace None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:54.122882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:54.199827 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3dffc8-ecc1-47d4-9014-df2034460ace None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:54.236892 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908621ba-3c8c-4dcd-b58f-b1155023598f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:54.237834 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908621ba-3c8c-4dcd-b58f-b1155023598f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:54.238184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908621ba-3c8c-4dcd-b58f-b1155023598f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:54.238466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908621ba-3c8c-4dcd-b58f-b1155023598f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:54.249272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908621ba-3c8c-4dcd-b58f-b1155023598f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:54.250949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908621ba-3c8c-4dcd-b58f-b1155023598f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-908621ba-3c8c-4dcd-b58f-b1155023598f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:14:54.266234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:54.267379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:14:54.268278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:14:54.268278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:14:54.268278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:14:54.268278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 462/933] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:14:54.274369 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf5574-3fed-4cfd-9200-51a58da6aa5e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:54.286869 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:54.376132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf5574-3fed-4cfd-9200-51a58da6aa5e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:54.412729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f55c9b-fa9c-4b57-84bd-5dc29c95a72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:54.413335 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f55c9b-fa9c-4b57-84bd-5dc29c95a72e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:54.413612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f55c9b-fa9c-4b57-84bd-5dc29c95a72e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:54.413883 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f55c9b-fa9c-4b57-84bd-5dc29c95a72e None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:54.414524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f55c9b-fa9c-4b57-84bd-5dc29c95a72e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:54.414825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f55c9b-fa9c-4b57-84bd-5dc29c95a72e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:54.416254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f55c9b-fa9c-4b57-84bd-5dc29c95a72e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:54.428041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 472/934] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:54.434151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908621ba-3c8c-4dcd-b58f-b1155023598f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:54.536449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:54.555598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908621ba-3c8c-4dcd-b58f-b1155023598f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:54.593526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee607-fa0e-430b-aef7-72c4059648b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:54.594239 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee607-fa0e-430b-aef7-72c4059648b1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:54.594526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee607-fa0e-430b-aef7-72c4059648b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:54.594822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee607-fa0e-430b-aef7-72c4059648b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:54.602348 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ee607-fa0e-430b-aef7-72c4059648b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:54.604498 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ee607-fa0e-430b-aef7-72c4059648b1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d94ee607-fa0e-430b-aef7-72c4059648b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:14:54.617016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:54.618326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:54.619301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:14:54.619301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:14:54.619301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:14:54.619301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:14:54.619301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:14:54.619301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:14:54.619301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:14:54.619301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:14:54.619301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 463/935] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:14:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:14:54.625299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f55c9b-fa9c-4b57-84bd-5dc29c95a72e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:54.708628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:54.748203 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f55c9b-fa9c-4b57-84bd-5dc29c95a72e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:54.783277 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150ad2c-a92b-42c1-b848-1f8229034f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:54.783885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ad2c-a92b-42c1-b848-1f8229034f00 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:54.784166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ad2c-a92b-42c1-b848-1f8229034f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:54.784441 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ad2c-a92b-42c1-b848-1f8229034f00 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:54.785129 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150ad2c-a92b-42c1-b848-1f8229034f00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:54.785435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150ad2c-a92b-42c1-b848-1f8229034f00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:54.787747 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150ad2c-a92b-42c1-b848-1f8229034f00 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:54.799509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 473/936] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:14:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:54.805925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee607-fa0e-430b-aef7-72c4059648b1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:54.910114 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee607-fa0e-430b-aef7-72c4059648b1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:54.949052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6ecdd-d1ff-4cac-8bc9-09c0a9a2a27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:54.949707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6ecdd-d1ff-4cac-8bc9-09c0a9a2a27e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:54.949984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6ecdd-d1ff-4cac-8bc9-09c0a9a2a27e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:54.950288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6ecdd-d1ff-4cac-8bc9-09c0a9a2a27e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:54.951007 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6ecdd-d1ff-4cac-8bc9-09c0a9a2a27e None admin] RBAC: Adding query filter params (role.id=8adf2afafb0446a6992e9aeb380f645d, scope.project.id=f28d337589024cc88184355bd0ee8572, user.id=5e935ba63cdd4256874b99a2eb0a1a2c) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:54.951346 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6ecdd-d1ff-4cac-8bc9-09c0a9a2a27e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:54.952766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6ecdd-d1ff-4cac-8bc9-09c0a9a2a27e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:54.959416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 464/937] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:14:54 2022] GET /identity/v3/role_assignments?user.id=5e935ba63cdd4256874b99a2eb0a1a2c&scope.project.id=f28d337589024cc88184355bd0ee8572&role.id=8adf2afafb0446a6992e9aeb380f645d => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:57.295760 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6532187-41c0-4a75-a70d-24b01a66f9d9 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:57.296316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6532187-41c0-4a75-a70d-24b01a66f9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:57.296316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6532187-41c0-4a75-a70d-24b01a66f9d9 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:57.297789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 474/938] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:14:57.305338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d749282-66ea-4d97-9085-b045599b4ebf None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:57.305791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d749282-66ea-4d97-9085-b045599b4ebf None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:57.305988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d749282-66ea-4d97-9085-b045599b4ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:57.325854 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-2d749282-66ea-4d97-9085-b045599b4ebf None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:14:57.327174 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2d749282-66ea-4d97-9085-b045599b4ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:14:57.330477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:57.403401 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 465/939] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:14:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:14:57.736409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6532187-41c0-4a75-a70d-24b01a66f9d9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:57.852678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6532187-41c0-4a75-a70d-24b01a66f9d9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:57.882484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:57.889282 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a4330-a9f5-49e0-bfe0-e731d570b508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:57.889946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a4330-a9f5-49e0-bfe0-e731d570b508 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:57.890225 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a4330-a9f5-49e0-bfe0-e731d570b508 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:57.890498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a4330-a9f5-49e0-bfe0-e731d570b508 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:57.901311 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a4330-a9f5-49e0-bfe0-e731d570b508 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-0a6a4330-a9f5-49e0-bfe0-e731d570b508 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:14:57.906079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:57.907122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:57.908121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:57.908121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:57.908121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:57.908121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:57.908121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:14:57.908121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:14:57.908121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:14:57.908121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:14:57.908121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:14:57.908121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:14:57.908121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:14:57.908121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:14:57.908121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 475/940] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:14:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:14:57.913908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d749282-66ea-4d97-9085-b045599b4ebf None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:58.026958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d749282-66ea-4d97-9085-b045599b4ebf None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:58.064929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393228e2-ab07-46ea-a940-e8cd170620f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:58.065581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393228e2-ab07-46ea-a940-e8cd170620f7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:58.065861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393228e2-ab07-46ea-a940-e8cd170620f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:58.066166 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393228e2-ab07-46ea-a940-e8cd170620f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:58.067128 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393228e2-ab07-46ea-a940-e8cd170620f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:58.067616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393228e2-ab07-46ea-a940-e8cd170620f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:58.069167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393228e2-ab07-46ea-a940-e8cd170620f7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:58.083686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 466/941] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:14:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:58.085247 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a4330-a9f5-49e0-bfe0-e731d570b508 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:58.205344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a4330-a9f5-49e0-bfe0-e731d570b508 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:58.248654 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff31696-5eb1-4efc-b02a-7800ef5f2a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:58.249347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff31696-5eb1-4efc-b02a-7800ef5f2a60 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:58.249688 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff31696-5eb1-4efc-b02a-7800ef5f2a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:58.250030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff31696-5eb1-4efc-b02a-7800ef5f2a60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:58.263877 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff31696-5eb1-4efc-b02a-7800ef5f2a60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:58.265400 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff31696-5eb1-4efc-b02a-7800ef5f2a60 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-7ff31696-5eb1-4efc-b02a-7800ef5f2a60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:14:58.281191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:58.282089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:14:58.282961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:14:58.282961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:14:58.282961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:14:58.282961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 476/942] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:14:58.289207 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393228e2-ab07-46ea-a940-e8cd170620f7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:58.292054 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:58.406569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393228e2-ab07-46ea-a940-e8cd170620f7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:58.447791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed376936-3d5c-4ce0-84d0-f5e456d3ca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:58.448549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed376936-3d5c-4ce0-84d0-f5e456d3ca7d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:58.448887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed376936-3d5c-4ce0-84d0-f5e456d3ca7d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:58.449444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed376936-3d5c-4ce0-84d0-f5e456d3ca7d None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:58.450444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed376936-3d5c-4ce0-84d0-f5e456d3ca7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:58.450982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed376936-3d5c-4ce0-84d0-f5e456d3ca7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:58.452792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed376936-3d5c-4ce0-84d0-f5e456d3ca7d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:58.470199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 467/943] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:58.477055 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff31696-5eb1-4efc-b02a-7800ef5f2a60 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:58.531890 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:58.598305 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff31696-5eb1-4efc-b02a-7800ef5f2a60 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:58.634009 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1edfa8-b796-41e8-a77f-60192aab0f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:58.634660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1edfa8-b796-41e8-a77f-60192aab0f43 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:58.634942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1edfa8-b796-41e8-a77f-60192aab0f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:58.635244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1edfa8-b796-41e8-a77f-60192aab0f43 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:58.643200 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1edfa8-b796-41e8-a77f-60192aab0f43 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:58.644651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1edfa8-b796-41e8-a77f-60192aab0f43 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-1c1edfa8-b796-41e8-a77f-60192aab0f43 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:14:58.655933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:58.657039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:58.658036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:58.658036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:58.658036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:14:58.658036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:14:58.658036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:14:58.658036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:14:58.658036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:14:58.658036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:14:58.658036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:14:58.658036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:14:58.658036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 477/944] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:14:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:14:58.663248 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed376936-3d5c-4ce0-84d0-f5e456d3ca7d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:58.728188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:58.771590 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed376936-3d5c-4ce0-84d0-f5e456d3ca7d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:58.811550 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb64792-a290-416d-ad3c-2ee15ba7f642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:58.812138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb64792-a290-416d-ad3c-2ee15ba7f642 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:58.812507 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb64792-a290-416d-ad3c-2ee15ba7f642 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:58.812768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb64792-a290-416d-ad3c-2ee15ba7f642 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:58.813588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb64792-a290-416d-ad3c-2ee15ba7f642 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:58.813926 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb64792-a290-416d-ad3c-2ee15ba7f642 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:58.815583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb64792-a290-416d-ad3c-2ee15ba7f642 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:58.836867 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 468/945] 10.222.0.18 () {56 vars in 1143 bytes} [Fri Oct 14 02:14:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:58.843340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1edfa8-b796-41e8-a77f-60192aab0f43 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:58.956638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1edfa8-b796-41e8-a77f-60192aab0f43 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:58.959019 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:58.996477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac049742-c82e-4d92-a7dd-3d38e5e63565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:58.997181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac049742-c82e-4d92-a7dd-3d38e5e63565 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:58.997485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac049742-c82e-4d92-a7dd-3d38e5e63565 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:58.997777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac049742-c82e-4d92-a7dd-3d38e5e63565 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:59.010772 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac049742-c82e-4d92-a7dd-3d38e5e63565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:59.012332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac049742-c82e-4d92-a7dd-3d38e5e63565 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-ac049742-c82e-4d92-a7dd-3d38e5e63565 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:14:59.027343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:59.028768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:14:59.029800 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:14:59.029800 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:14:59.029800 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:14:59.029800 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 478/946] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:14:59.034387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb64792-a290-416d-ad3c-2ee15ba7f642 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:59.151652 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb64792-a290-416d-ad3c-2ee15ba7f642 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:59.174818 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:59.190847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abc635-9593-47b8-9eab-168c9269f658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:59.191643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abc635-9593-47b8-9eab-168c9269f658 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:59.192098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abc635-9593-47b8-9eab-168c9269f658 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:59.192585 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abc635-9593-47b8-9eab-168c9269f658 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:59.193475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abc635-9593-47b8-9eab-168c9269f658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:59.193933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abc635-9593-47b8-9eab-168c9269f658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:59.195557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abc635-9593-47b8-9eab-168c9269f658 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:59.209660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 469/947] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:59.216386 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac049742-c82e-4d92-a7dd-3d38e5e63565 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:59.331924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac049742-c82e-4d92-a7dd-3d38e5e63565 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:59.368267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe9d64-629a-4e04-8b8c-2510aaaab583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:59.371625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9d64-629a-4e04-8b8c-2510aaaab583 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:59.371625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9d64-629a-4e04-8b8c-2510aaaab583 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:59.371625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9d64-629a-4e04-8b8c-2510aaaab583 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:59.377478 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe9d64-629a-4e04-8b8c-2510aaaab583 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:59.379166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe9d64-629a-4e04-8b8c-2510aaaab583 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-71fe9d64-629a-4e04-8b8c-2510aaaab583 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:14:59.394198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:14:59.395346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:14:59.396215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:14:59.396215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:14:59.396215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:14:59.396215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:14:59.396215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:14:59.396215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:14:59.396215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:14:59.396215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:14:59.396215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 479/948] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:14:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:14:59.402552 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abc635-9593-47b8-9eab-168c9269f658 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:59.523623 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abc635-9593-47b8-9eab-168c9269f658 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:59.564351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd22b3-e6c7-4a3c-8513-b4ca245ddba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:59.565027 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd22b3-e6c7-4a3c-8513-b4ca245ddba5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:59.565344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd22b3-e6c7-4a3c-8513-b4ca245ddba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:59.565681 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd22b3-e6c7-4a3c-8513-b4ca245ddba5 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:59.566559 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd22b3-e6c7-4a3c-8513-b4ca245ddba5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:14:59.566879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd22b3-e6c7-4a3c-8513-b4ca245ddba5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:59.568531 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd22b3-e6c7-4a3c-8513-b4ca245ddba5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:59.583173 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 470/949] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:14:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:14:59.590028 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe9d64-629a-4e04-8b8c-2510aaaab583 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:14:59.609370 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:14:59.708418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe9d64-629a-4e04-8b8c-2510aaaab583 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:14:59.749671 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27a0f5-3af9-4372-b1e5-bfd9fe5ec90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:14:59.750358 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27a0f5-3af9-4372-b1e5-bfd9fe5ec90b None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:14:59.750729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27a0f5-3af9-4372-b1e5-bfd9fe5ec90b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:14:59.751005 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27a0f5-3af9-4372-b1e5-bfd9fe5ec90b None admin] PATH_INFO: `/v3/projects/f28d337589024cc88184355bd0ee8572/users/5e935ba63cdd4256874b99a2eb0a1a2c/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:14:59.753201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27a0f5-3af9-4372-b1e5-bfd9fe5ec90b None admin] RBAC: Authorizing `identity:create_grant(project_id=f28d337589024cc88184355bd0ee8572, user_id=5e935ba63cdd4256874b99a2eb0a1a2c, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:14:59.755297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27a0f5-3af9-4372-b1e5-bfd9fe5ec90b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:14:59.769697 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 480/950] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Oct 14 02:14:59 2022] PUT /identity/v3/projects/f28d337589024cc88184355bd0ee8572/users/5e935ba63cdd4256874b99a2eb0a1a2c/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:15:02.053217 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a1388-739a-4559-bea8-6f82055f95f4 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:02.053945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a1388-739a-4559-bea8-6f82055f95f4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:02.053945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a1388-739a-4559-bea8-6f82055f95f4 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:02.055475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 471/951] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:15:02.062087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae8036-9acb-4e46-9a5a-3b7fea576159 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:02.062502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae8036-9acb-4e46-9a5a-3b7fea576159 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:02.062592 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae8036-9acb-4e46-9a5a-3b7fea576159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:02.082953 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-67ae8036-9acb-4e46-9a5a-3b7fea576159 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:15:02.084272 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-67ae8036-9acb-4e46-9a5a-3b7fea576159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:15:02.154568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 481/952] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:15:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:15:02.468376 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a1388-739a-4559-bea8-6f82055f95f4 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:02.521899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:02.597219 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a1388-739a-4559-bea8-6f82055f95f4 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:02.635314 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5447ec-8f27-410f-8c57-e29c9f4149d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:02.636093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5447ec-8f27-410f-8c57-e29c9f4149d2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:02.636343 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5447ec-8f27-410f-8c57-e29c9f4149d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:02.636669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5447ec-8f27-410f-8c57-e29c9f4149d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:02.648533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5447ec-8f27-410f-8c57-e29c9f4149d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-3d5447ec-8f27-410f-8c57-e29c9f4149d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:15:02.653759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:15:02.654769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:15:02.655908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:15:02.655908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:15:02.655908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:15:02.655908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:15:02.655908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:15:02.655908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:15:02.655908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:15:02.655908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:15:02.655908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:15:02.655908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:15:02.655908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:15:02.655908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:15:02.655908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 472/953] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Oct 14 02:15:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:15:02.661819 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae8036-9acb-4e46-9a5a-3b7fea576159 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:02.665186 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:02.779731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae8036-9acb-4e46-9a5a-3b7fea576159 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:02.819180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec411580-fbc3-4611-bc81-45a8435c48f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:02.819788 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec411580-fbc3-4611-bc81-45a8435c48f9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:02.820064 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec411580-fbc3-4611-bc81-45a8435c48f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:02.820334 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec411580-fbc3-4611-bc81-45a8435c48f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:02.820989 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec411580-fbc3-4611-bc81-45a8435c48f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:15:02.821288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec411580-fbc3-4611-bc81-45a8435c48f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:02.822640 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec411580-fbc3-4611-bc81-45a8435c48f9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:02.831114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 482/954] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Oct 14 02:15:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:15:02.837667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5447ec-8f27-410f-8c57-e29c9f4149d2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:02.948245 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:02.951687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5447ec-8f27-410f-8c57-e29c9f4149d2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:02.988854 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ae683-7b92-434a-b05a-ffc231487d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:02.989570 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ae683-7b92-434a-b05a-ffc231487d6f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:02.989853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ae683-7b92-434a-b05a-ffc231487d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:02.990123 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ae683-7b92-434a-b05a-ffc231487d6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:03.001388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ae683-7b92-434a-b05a-ffc231487d6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:03.002821 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ae683-7b92-434a-b05a-ffc231487d6f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-e63ae683-7b92-434a-b05a-ffc231487d6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:15:03.017806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:15:03.018797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:15:03.019830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:15:03.019830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:15:03.019830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:15:03.019830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 473/955] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:15:03.024815 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec411580-fbc3-4611-bc81-45a8435c48f9 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:03.097211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:03.137704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec411580-fbc3-4611-bc81-45a8435c48f9 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:03.176461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e68e5d-c0e5-4db9-9e4c-c10c02628fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:03.177105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e68e5d-c0e5-4db9-9e4c-c10c02628fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:03.177384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e68e5d-c0e5-4db9-9e4c-c10c02628fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:03.177660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e68e5d-c0e5-4db9-9e4c-c10c02628fe4 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:03.178328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e68e5d-c0e5-4db9-9e4c-c10c02628fe4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:15:03.178629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e68e5d-c0e5-4db9-9e4c-c10c02628fe4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:03.180097 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e68e5d-c0e5-4db9-9e4c-c10c02628fe4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:03.196019 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 483/956] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:15:03.201783 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ae683-7b92-434a-b05a-ffc231487d6f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:03.308012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ae683-7b92-434a-b05a-ffc231487d6f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:03.343275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102747ba-fc64-4c0f-812b-ccc35754b1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:03.343928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102747ba-fc64-4c0f-812b-ccc35754b1ae None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:03.344209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102747ba-fc64-4c0f-812b-ccc35754b1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:03.344482 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102747ba-fc64-4c0f-812b-ccc35754b1ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:03.354198 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102747ba-fc64-4c0f-812b-ccc35754b1ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:03.355796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102747ba-fc64-4c0f-812b-ccc35754b1ae None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-102747ba-fc64-4c0f-812b-ccc35754b1ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:15:03.369485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:15:03.370603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:15:03.371635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:15:03.371635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:15:03.371635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:15:03.371635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:15:03.371635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:15:03.371635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:15:03.371635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:15:03.371635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:15:03.371635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:15:03.371635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:15:03.371635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 474/957] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:15:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:15:03.376727 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e68e5d-c0e5-4db9-9e4c-c10c02628fe4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:03.491474 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e68e5d-c0e5-4db9-9e4c-c10c02628fe4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:03.520116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:03.529791 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3eedd0-04a2-4a24-885f-6ddee0a095b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:03.530405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3eedd0-04a2-4a24-885f-6ddee0a095b4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:03.530680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3eedd0-04a2-4a24-885f-6ddee0a095b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:03.530948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3eedd0-04a2-4a24-885f-6ddee0a095b4 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:03.531708 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3eedd0-04a2-4a24-885f-6ddee0a095b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:15:03.532012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3eedd0-04a2-4a24-885f-6ddee0a095b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:03.533438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3eedd0-04a2-4a24-885f-6ddee0a095b4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:03.554148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 484/958] 10.222.0.18 () {56 vars in 1143 bytes} [Fri Oct 14 02:15:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:15:03.560172 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102747ba-fc64-4c0f-812b-ccc35754b1ae None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:03.576289 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:03.668787 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102747ba-fc64-4c0f-812b-ccc35754b1ae None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:03.701183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49a456-adf5-4a29-a925-9b8895a2a8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:03.701793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49a456-adf5-4a29-a925-9b8895a2a8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:03.702066 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49a456-adf5-4a29-a925-9b8895a2a8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:03.702335 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49a456-adf5-4a29-a925-9b8895a2a8b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:03.773516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49a456-adf5-4a29-a925-9b8895a2a8b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:03.774974 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49a456-adf5-4a29-a925-9b8895a2a8b2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:03.788601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:03.874089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 475/959] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Oct 14 02:15:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:15:03.880384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3eedd0-04a2-4a24-885f-6ddee0a095b4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:03.975669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3eedd0-04a2-4a24-885f-6ddee0a095b4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:04.008379 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e33a8-e5b7-4100-82cf-bfb5fae84d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:04.008987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e33a8-e5b7-4100-82cf-bfb5fae84d2e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:04.009262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e33a8-e5b7-4100-82cf-bfb5fae84d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:04.009530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e33a8-e5b7-4100-82cf-bfb5fae84d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:04.018964 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e33a8-e5b7-4100-82cf-bfb5fae84d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:04.020515 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e33a8-e5b7-4100-82cf-bfb5fae84d2e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-848e33a8-e5b7-4100-82cf-bfb5fae84d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:15:04.034390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:15:04.036148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:15:04.037232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:15:04.037232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:15:04.037232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:15:04.037232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 485/960] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:15:04.041763 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49a456-adf5-4a29-a925-9b8895a2a8b2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:04.136201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49a456-adf5-4a29-a925-9b8895a2a8b2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:04.148775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:04.169334 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb736b0-b4cb-4857-a4ed-2d83b4a4f198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:04.169933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb736b0-b4cb-4857-a4ed-2d83b4a4f198 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:04.170211 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb736b0-b4cb-4857-a4ed-2d83b4a4f198 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:04.170483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb736b0-b4cb-4857-a4ed-2d83b4a4f198 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:04.171145 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb736b0-b4cb-4857-a4ed-2d83b4a4f198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:15:04.171447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb736b0-b4cb-4857-a4ed-2d83b4a4f198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:04.172800 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb736b0-b4cb-4857-a4ed-2d83b4a4f198 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:04.183550 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 476/961] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Oct 14 02:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:15:04.189052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e33a8-e5b7-4100-82cf-bfb5fae84d2e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:04.224075 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:04.290217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e33a8-e5b7-4100-82cf-bfb5fae84d2e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:04.324247 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613fd0f-5d72-4ea9-a4ba-36814f77076d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:04.324864 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613fd0f-5d72-4ea9-a4ba-36814f77076d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:04.325183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613fd0f-5d72-4ea9-a4ba-36814f77076d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:04.325457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613fd0f-5d72-4ea9-a4ba-36814f77076d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:04.332792 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613fd0f-5d72-4ea9-a4ba-36814f77076d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:04.334324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613fd0f-5d72-4ea9-a4ba-36814f77076d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-2613fd0f-5d72-4ea9-a4ba-36814f77076d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:15:04.345596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:15:04.346596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:15:04.347598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:15:04.347598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:15:04.347598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:15:04.347598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:15:04.347598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:15:04.347598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:15:04.347598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:15:04.347598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 14 02:15:04.347598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 486/962] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Oct 14 02:15:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:15:04.352460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb736b0-b4cb-4857-a4ed-2d83b4a4f198 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:04.454038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb736b0-b4cb-4857-a4ed-2d83b4a4f198 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:04.490905 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6b256-a1ad-4ee3-9fd7-8702c07517f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:04.491546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6b256-a1ad-4ee3-9fd7-8702c07517f0 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:04.491823 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6b256-a1ad-4ee3-9fd7-8702c07517f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:04.492345 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6b256-a1ad-4ee3-9fd7-8702c07517f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:04.493032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6b256-a1ad-4ee3-9fd7-8702c07517f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:15:04.493337 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6b256-a1ad-4ee3-9fd7-8702c07517f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:04.494756 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6b256-a1ad-4ee3-9fd7-8702c07517f0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:04.506867 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 477/963] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Oct 14 02:15:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:15:04.513049 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613fd0f-5d72-4ea9-a4ba-36814f77076d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:04.604564 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:04.610815 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613fd0f-5d72-4ea9-a4ba-36814f77076d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:04.645138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c48e2-60b1-448c-a554-e5b061b7d3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:04.645741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c48e2-60b1-448c-a554-e5b061b7d3fb None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:04.646022 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c48e2-60b1-448c-a554-e5b061b7d3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:04.646296 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c48e2-60b1-448c-a554-e5b061b7d3fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:04.647017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c48e2-60b1-448c-a554-e5b061b7d3fb None admin] RBAC: Adding query filter params (role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b, scope.project.id=f28d337589024cc88184355bd0ee8572, user.id=5e935ba63cdd4256874b99a2eb0a1a2c) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:15:04.647356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c48e2-60b1-448c-a554-e5b061b7d3fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:04.648777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c48e2-60b1-448c-a554-e5b061b7d3fb None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:04.654403 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 487/964] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Oct 14 02:15:04 2022] GET /identity/v3/role_assignments?user.id=5e935ba63cdd4256874b99a2eb0a1a2c&scope.project.id=f28d337589024cc88184355bd0ee8572&role.id=9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:15:06.827554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8a1d3-dd7f-43da-9ca3-587c8988b03d None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:06.828066 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8a1d3-dd7f-43da-9ca3-587c8988b03d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:06.828175 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8a1d3-dd7f-43da-9ca3-587c8988b03d None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:06.829421 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 478/965] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:15:06.834826 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5831f-16b0-4bc2-987f-7188f2dd64fc None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:06.835267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5831f-16b0-4bc2-987f-7188f2dd64fc None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:06.835423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5831f-16b0-4bc2-987f-7188f2dd64fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:06.853201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1bb5831f-16b0-4bc2-987f-7188f2dd64fc None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:15:06.854377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1bb5831f-16b0-4bc2-987f-7188f2dd64fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:15:06.918791 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 488/966] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:15:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:15:07.185463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8a1d3-dd7f-43da-9ca3-587c8988b03d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:07.216449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:07.285280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8a1d3-dd7f-43da-9ca3-587c8988b03d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:07.318246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447299a8-4c1b-4ad6-8498-45a41f61bdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:07.318853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447299a8-4c1b-4ad6-8498-45a41f61bdbe None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:07.319151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447299a8-4c1b-4ad6-8498-45a41f61bdbe None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:07.319422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447299a8-4c1b-4ad6-8498-45a41f61bdbe None admin] PATH_INFO: `/v3/services/placement` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:07.319991 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447299a8-4c1b-4ad6-8498-45a41f61bdbe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:07.321336 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447299a8-4c1b-4ad6-8498-45a41f61bdbe None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-447299a8-4c1b-4ad6-8498-45a41f61bdbe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:15:07.329778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:15:07.330957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:15:07.332416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 479/967] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Oct 14 02:15:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:15:07.337834 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5831f-16b0-4bc2-987f-7188f2dd64fc None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:07.422831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:07.435842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5831f-16b0-4bc2-987f-7188f2dd64fc None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:07.475831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9562a52-85f2-41b4-b6b0-3236ac9f0921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:07.476428 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9562a52-85f2-41b4-b6b0-3236ac9f0921 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:07.476716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9562a52-85f2-41b4-b6b0-3236ac9f0921 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:07.477031 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9562a52-85f2-41b4-b6b0-3236ac9f0921 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:07.477707 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9562a52-85f2-41b4-b6b0-3236ac9f0921 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:15:07.478083 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9562a52-85f2-41b4-b6b0-3236ac9f0921 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:07.479792 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9562a52-85f2-41b4-b6b0-3236ac9f0921 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:07.484844 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 489/968] 10.222.0.18 () {56 vars in 1116 bytes} [Fri Oct 14 02:15:07 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:15:07.492566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447299a8-4c1b-4ad6-8498-45a41f61bdbe None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:07.585770 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:07.592110 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447299a8-4c1b-4ad6-8498-45a41f61bdbe None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:07.624896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d06244-9768-4883-b73b-da6818416b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:07.625495 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d06244-9768-4883-b73b-da6818416b0e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:07.625776 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d06244-9768-4883-b73b-da6818416b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:07.626050 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d06244-9768-4883-b73b-da6818416b0e None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:07.626673 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d06244-9768-4883-b73b-da6818416b0e None admin] RBAC: Adding query filter params (type=placement) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:15:07.626975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d06244-9768-4883-b73b-da6818416b0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:07.628366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d06244-9768-4883-b73b-da6818416b0e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:07.633469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 480/969] 10.222.0.18 () {56 vars in 1116 bytes} [Fri Oct 14 02:15:07 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:15:09.772313 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0382b-08c0-4239-b88b-402f422e562f None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:09.772845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0382b-08c0-4239-b88b-402f422e562f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:09.772845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0382b-08c0-4239-b88b-402f422e562f None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:09.774193 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 490/970] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:15:09.779416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf937386-0f4c-4a01-ac82-c2d685251ba6 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:09.779847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf937386-0f4c-4a01-ac82-c2d685251ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:09.779944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf937386-0f4c-4a01-ac82-c2d685251ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:09.797894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-cf937386-0f4c-4a01-ac82-c2d685251ba6 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:15:09.799085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cf937386-0f4c-4a01-ac82-c2d685251ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:15:09.863689 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 481/971] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:15:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:15:10.134117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0382b-08c0-4239-b88b-402f422e562f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:10.232758 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0382b-08c0-4239-b88b-402f422e562f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:10.265858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669d23e-2bcf-4b1b-83f2-8982ef475f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:10.266464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669d23e-2bcf-4b1b-83f2-8982ef475f09 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:10.266741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669d23e-2bcf-4b1b-83f2-8982ef475f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:10.267006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669d23e-2bcf-4b1b-83f2-8982ef475f09 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:10.267697 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669d23e-2bcf-4b1b-83f2-8982ef475f09 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:10.269634 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669d23e-2bcf-4b1b-83f2-8982ef475f09 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:10.283683 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 491/972] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Oct 14 02:15:10 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:15:12.418413 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636629bf-655a-4d11-bbc3-f333535a4090 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:12.418966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636629bf-655a-4d11-bbc3-f333535a4090 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:12.418966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636629bf-655a-4d11-bbc3-f333535a4090 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:12.420366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 482/973] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:15:12.426205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf94ff1-adb1-45ce-b18d-bfe70e7a31f1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:12.426692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf94ff1-adb1-45ce-b18d-bfe70e7a31f1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:12.427101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf94ff1-adb1-45ce-b18d-bfe70e7a31f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:12.448607 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4cf94ff1-adb1-45ce-b18d-bfe70e7a31f1 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:15:12.449732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4cf94ff1-adb1-45ce-b18d-bfe70e7a31f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:15:12.461513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:12.533125 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 492/974] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:15:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:15:12.797291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636629bf-655a-4d11-bbc3-f333535a4090 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:12.886092 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:12.896716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636629bf-655a-4d11-bbc3-f333535a4090 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:12.929844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5b5eb-422c-46bf-982b-f4b66e91ed82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:12.930462 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5b5eb-422c-46bf-982b-f4b66e91ed82 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:12.930738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5b5eb-422c-46bf-982b-f4b66e91ed82 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:12.931007 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5b5eb-422c-46bf-982b-f4b66e91ed82 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:12.931606 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5b5eb-422c-46bf-982b-f4b66e91ed82 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:12.932963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5b5eb-422c-46bf-982b-f4b66e91ed82 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-41c5b5eb-422c-46bf-982b-f4b66e91ed82 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:15:12.942020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:15:12.943288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:15:12.944251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 483/975] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Oct 14 02:15:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:15:12.949187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf94ff1-adb1-45ce-b18d-bfe70e7a31f1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:13.042792 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf94ff1-adb1-45ce-b18d-bfe70e7a31f1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:13.074813 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c55994-ebc6-488c-95f1-820dbfa7a82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:13.075431 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c55994-ebc6-488c-95f1-820dbfa7a82a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:13.075706 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c55994-ebc6-488c-95f1-820dbfa7a82a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:13.075972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c55994-ebc6-488c-95f1-820dbfa7a82a None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:13.076587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c55994-ebc6-488c-95f1-820dbfa7a82a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:15:13.076885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c55994-ebc6-488c-95f1-820dbfa7a82a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:13.078235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c55994-ebc6-488c-95f1-820dbfa7a82a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:13.083212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 493/976] 10.222.0.18 () {56 vars in 1116 bytes} [Fri Oct 14 02:15:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:15:13.089095 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5b5eb-422c-46bf-982b-f4b66e91ed82 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:13.184786 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5b5eb-422c-46bf-982b-f4b66e91ed82 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:13.218261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f309dda-8ace-4648-801c-f116c22d4947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:13.218855 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f309dda-8ace-4648-801c-f116c22d4947 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:13.219156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f309dda-8ace-4648-801c-f116c22d4947 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:13.219427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f309dda-8ace-4648-801c-f116c22d4947 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:13.220114 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f309dda-8ace-4648-801c-f116c22d4947 None admin] RBAC: Adding query filter params (interface=public, service_id=10e8e90a9918497a8d9164fbb7c88c39, region_id=RegionOne) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:15:13.220418 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f309dda-8ace-4648-801c-f116c22d4947 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:13.221765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f309dda-8ace-4648-801c-f116c22d4947 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:13.226534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 484/977] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Oct 14 02:15:13 2022] GET /identity/v3/endpoints?service_id=10e8e90a9918497a8d9164fbb7c88c39&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:15:13.232237 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c55994-ebc6-488c-95f1-820dbfa7a82a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:13.235892 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:13.327974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c55994-ebc6-488c-95f1-820dbfa7a82a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:13.359759 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a4cf7-b132-4b34-85f6-4d897e36a43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:13.360349 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a4cf7-b132-4b34-85f6-4d897e36a43b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:13.360624 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a4cf7-b132-4b34-85f6-4d897e36a43b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:13.360892 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a4cf7-b132-4b34-85f6-4d897e36a43b None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:13.361479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a4cf7-b132-4b34-85f6-4d897e36a43b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:13.363399 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a4cf7-b132-4b34-85f6-4d897e36a43b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:13.370189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 494/978] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Oct 14 02:15:13 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:15:15.488676 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bdeac-a94d-492a-a64e-aa8169e4b9cd None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:15.489262 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bdeac-a94d-492a-a64e-aa8169e4b9cd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:15.489496 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bdeac-a94d-492a-a64e-aa8169e4b9cd None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:15.491089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 485/979] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:15:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:15:15.497325 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f962a-8546-4e37-a5a4-0cd46db3c980 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:15.497794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f962a-8546-4e37-a5a4-0cd46db3c980 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:15.497972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f962a-8546-4e37-a5a4-0cd46db3c980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:15.517989 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-816f962a-8546-4e37-a5a4-0cd46db3c980 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:15:15.519966 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-816f962a-8546-4e37-a5a4-0cd46db3c980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:15:15.584469 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:15.590154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 495/980] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:15:15 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:15:15.854464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bdeac-a94d-492a-a64e-aa8169e4b9cd None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:15.873924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:15.952400 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bdeac-a94d-492a-a64e-aa8169e4b9cd None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:15.984543 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1613f-c86b-427d-9844-f7dddc719218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:15.985159 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1613f-c86b-427d-9844-f7dddc719218 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:15.985436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1613f-c86b-427d-9844-f7dddc719218 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:15.985706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1613f-c86b-427d-9844-f7dddc719218 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:15.986272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1613f-c86b-427d-9844-f7dddc719218 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:15.987755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1613f-c86b-427d-9844-f7dddc719218 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d4b1613f-c86b-427d-9844-f7dddc719218 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:15:15.996374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:15:15.997546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 14 02:15:15.998381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 486/981] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Oct 14 02:15:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:15:16.003898 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f962a-8546-4e37-a5a4-0cd46db3c980 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:16.099820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f962a-8546-4e37-a5a4-0cd46db3c980 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:16.133583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb6910-4c22-478d-be3a-482d7c7bf608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:16.134122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb6910-4c22-478d-be3a-482d7c7bf608 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:16.134404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb6910-4c22-478d-be3a-482d7c7bf608 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:16.134679 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb6910-4c22-478d-be3a-482d7c7bf608 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:16.135335 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb6910-4c22-478d-be3a-482d7c7bf608 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:15:16.135640 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb6910-4c22-478d-be3a-482d7c7bf608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:16.137003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb6910-4c22-478d-be3a-482d7c7bf608 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:16.141918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 496/982] 10.222.0.18 () {56 vars in 1116 bytes} [Fri Oct 14 02:15:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:15:16.147737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1613f-c86b-427d-9844-f7dddc719218 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:15:16.226891 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:16.245405 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1613f-c86b-427d-9844-f7dddc719218 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:15:16.277850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d90dd9-a7d5-4212-94de-52181f8f3f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:15:16.278444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d90dd9-a7d5-4212-94de-52181f8f3f98 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:15:16.278722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d90dd9-a7d5-4212-94de-52181f8f3f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:15:16.278994 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d90dd9-a7d5-4212-94de-52181f8f3f98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:15:16.279687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d90dd9-a7d5-4212-94de-52181f8f3f98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:15:16.281046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d90dd9-a7d5-4212-94de-52181f8f3f98 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:15:16.300765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 487/983] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:15:16 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:15:26.139025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=69 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:15:26.295827 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:19.603488 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0d15a-31f1-4001-94cf-1fc55f1d5474 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:19.603972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0d15a-31f1-4001-94cf-1fc55f1d5474 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:19.604050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0d15a-31f1-4001-94cf-1fc55f1d5474 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:19.611516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e787ce4-9a47-48e3-8dcc-094bb7ed43a2 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:19.611926 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e787ce4-9a47-48e3-8dcc-094bb7ed43a2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:19.612136 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e787ce4-9a47-48e3-8dcc-094bb7ed43a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:19.616877 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 497/984] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:16:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:16:19.700450 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4e787ce4-9a47-48e3-8dcc-094bb7ed43a2 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:16:19.701619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4e787ce4-9a47-48e3-8dcc-094bb7ed43a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:19.821627 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 488/985] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:16:20.123101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad373f-854d-4708-88a7-b969a70cb003 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:20.123426 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad373f-854d-4708-88a7-b969a70cb003 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:20.123653 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad373f-854d-4708-88a7-b969a70cb003 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:20.125342 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 498/986] 10.222.0.18 () {54 vars in 964 bytes} [Fri Oct 14 02:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:16:20.131234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfca31-4ba2-484b-874f-49b2cd18f5e6 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:20.131523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfca31-4ba2-484b-874f-49b2cd18f5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:20.131798 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfca31-4ba2-484b-874f-49b2cd18f5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:20.215395 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-2bdfca31-4ba2-484b-874f-49b2cd18f5e6 None None] MFA Rules not processed for user `6ff451e43cea4bb0a5fd73491a05bf82`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:16:20.216842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2bdfca31-4ba2-484b-874f-49b2cd18f5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:20.239667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:20.303282 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 489/987] 10.222.0.18 () {58 vars in 1061 bytes} [Fri Oct 14 02:16:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:16:20.312067 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad373f-854d-4708-88a7-b969a70cb003 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:16:20.313912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-62ad373f-854d-4708-88a7-b969a70cb003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:20.441944 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad373f-854d-4708-88a7-b969a70cb003 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:16:20.542888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd69593-aa6e-4ace-bc89-379cf37f60ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:16:20.543792 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd69593-aa6e-4ace-bc89-379cf37f60ff service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:20.544213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd69593-aa6e-4ace-bc89-379cf37f60ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:20.544648 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd69593-aa6e-4ace-bc89-379cf37f60ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:20.546633 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6fd69593-aa6e-4ace-bc89-379cf37f60ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:20.610163 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:20.674362 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd69593-aa6e-4ace-bc89-379cf37f60ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:16:20.675985 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd69593-aa6e-4ace-bc89-379cf37f60ff service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:16:20.697003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 499/988] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:16:23.193139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a922e3a-4615-44cf-a6de-4176c22f62f8 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:23.193673 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a922e3a-4615-44cf-a6de-4176c22f62f8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:23.193673 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a922e3a-4615-44cf-a6de-4176c22f62f8 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:23.195037 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 490/989] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:16:23.200341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c623e98-02bb-4b4b-a090-cb14561cf193 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:23.200790 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c623e98-02bb-4b4b-a090-cb14561cf193 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:23.200884 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c623e98-02bb-4b4b-a090-cb14561cf193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:23.275910 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6c623e98-02bb-4b4b-a090-cb14561cf193 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:16:23.277139 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6c623e98-02bb-4b4b-a090-cb14561cf193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:23.377916 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 500/990] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:16:23.663749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a922e3a-4615-44cf-a6de-4176c22f62f8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:16:23.665289 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2a922e3a-4615-44cf-a6de-4176c22f62f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:23.769725 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a922e3a-4615-44cf-a6de-4176c22f62f8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:16:23.840625 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:23.843221 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad500a-3405-4be1-b0be-891888743e77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:16:23.844059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad500a-3405-4be1-b0be-891888743e77 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:23.844521 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad500a-3405-4be1-b0be-891888743e77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:23.844978 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad500a-3405-4be1-b0be-891888743e77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:23.846589 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e4ad500a-3405-4be1-b0be-891888743e77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:23.955717 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad500a-3405-4be1-b0be-891888743e77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:16:23.957070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad500a-3405-4be1-b0be-891888743e77 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:16:23.974542 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 491/991] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:16:26.405526 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cb6bb-df65-4724-865a-ff6db1e8f993 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:26.406060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cb6bb-df65-4724-865a-ff6db1e8f993 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:26.406060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cb6bb-df65-4724-865a-ff6db1e8f993 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:26.407435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 501/992] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:16:26.412578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43e4a1-5dca-44aa-aba0-4b0bfe9b3191 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:26.412991 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43e4a1-5dca-44aa-aba0-4b0bfe9b3191 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:26.413131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43e4a1-5dca-44aa-aba0-4b0bfe9b3191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:26.480264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ce43e4a1-5dca-44aa-aba0-4b0bfe9b3191 None None] MFA Rules not processed for user `e989ab80fa194e5ca4a4e53f88ae8ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:16:26.481485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ce43e4a1-5dca-44aa-aba0-4b0bfe9b3191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:26.533778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:26.578124 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 492/993] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:16:26 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:16:26.873873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cb6bb-df65-4724-865a-ff6db1e8f993 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:16:26.875440 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f90cb6bb-df65-4724-865a-ff6db1e8f993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:26.887562 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:27.002755 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cb6bb-df65-4724-865a-ff6db1e8f993 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:16:27.087670 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db0895-f1b3-4787-bb5b-3190f4287746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:16:27.088282 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db0895-f1b3-4787-bb5b-3190f4287746 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:27.088561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db0895-f1b3-4787-bb5b-3190f4287746 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:27.088834 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db0895-f1b3-4787-bb5b-3190f4287746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:27.090435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-99db0895-f1b3-4787-bb5b-3190f4287746 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:27.156558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:27.206088 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db0895-f1b3-4787-bb5b-3190f4287746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:16:27.207572 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db0895-f1b3-4787-bb5b-3190f4287746 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:16:27.226685 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 502/994] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:16:26 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:16:29.926354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff8922-a25b-4bcf-b894-e9b3e6f4ba81 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:29.926902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff8922-a25b-4bcf-b894-e9b3e6f4ba81 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:29.926902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff8922-a25b-4bcf-b894-e9b3e6f4ba81 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:29.928255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 493/995] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:16:29.933418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002cded-2d94-45be-9fb3-82fefcf82d99 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:29.933841 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002cded-2d94-45be-9fb3-82fefcf82d99 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:29.933961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002cded-2d94-45be-9fb3-82fefcf82d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:29.998515 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2002cded-2d94-45be-9fb3-82fefcf82d99 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:16:29.999724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2002cded-2d94-45be-9fb3-82fefcf82d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:30.093263 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 503/996] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:16:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:16:30.389265 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e71790-54fc-4b1f-8de5-ba39903f8c0b None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:30.389546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e71790-54fc-4b1f-8de5-ba39903f8c0b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:30.389816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e71790-54fc-4b1f-8de5-ba39903f8c0b None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:30.391191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 494/997] 10.222.0.18 () {54 vars in 964 bytes} [Fri Oct 14 02:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:16:30.396927 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef2899-0aed-4591-af16-ea8d5218febb None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:30.397220 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef2899-0aed-4591-af16-ea8d5218febb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:30.397502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef2899-0aed-4591-af16-ea8d5218febb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:30.419914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:30.480827 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d4ef2899-0aed-4591-af16-ea8d5218febb None None] MFA Rules not processed for user `6ff451e43cea4bb0a5fd73491a05bf82`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:16:30.482062 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d4ef2899-0aed-4591-af16-ea8d5218febb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:30.556848 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 504/998] 10.222.0.18 () {58 vars in 1061 bytes} [Fri Oct 14 02:16:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:16:30.565904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e71790-54fc-4b1f-8de5-ba39903f8c0b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:16:30.567477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f6e71790-54fc-4b1f-8de5-ba39903f8c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:30.668417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e71790-54fc-4b1f-8de5-ba39903f8c0b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:16:30.739769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89137b97-2b94-4ce3-8a6e-cf3d6d3d72e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:16:30.740379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89137b97-2b94-4ce3-8a6e-cf3d6d3d72e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:30.740657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89137b97-2b94-4ce3-8a6e-cf3d6d3d72e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:30.740928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89137b97-2b94-4ce3-8a6e-cf3d6d3d72e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:30.742509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-89137b97-2b94-4ce3-8a6e-cf3d6d3d72e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:30.749538 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:30.849976 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89137b97-2b94-4ce3-8a6e-cf3d6d3d72e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:16:30.851452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89137b97-2b94-4ce3-8a6e-cf3d6d3d72e3 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:16:30.869951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 495/999] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:16:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:16:34.119041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1845f0f-db11-46e2-9eda-164d801afb32 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:34.119625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1845f0f-db11-46e2-9eda-164d801afb32 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:34.119625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1845f0f-db11-46e2-9eda-164d801afb32 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:34.120975 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 505/1000] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:16:34.126244 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a0218-f9e9-4a05-a5ea-c832c5875194 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:34.126628 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a0218-f9e9-4a05-a5ea-c832c5875194 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:34.126802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a0218-f9e9-4a05-a5ea-c832c5875194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:34.199826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-581a0218-f9e9-4a05-a5ea-c832c5875194 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:16:34.201095 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-581a0218-f9e9-4a05-a5ea-c832c5875194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:34.263101 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:34.304793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 496/1001] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:16:34.588665 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1845f0f-db11-46e2-9eda-164d801afb32 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:16:34.590180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c1845f0f-db11-46e2-9eda-164d801afb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:34.695703 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1845f0f-db11-46e2-9eda-164d801afb32 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:16:34.708373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:34.777267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7adb9f3-3c6d-47de-925e-43846a0f6436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:16:34.777878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adb9f3-3c6d-47de-925e-43846a0f6436 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:34.778159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adb9f3-3c6d-47de-925e-43846a0f6436 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:34.778433 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adb9f3-3c6d-47de-925e-43846a0f6436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:34.780702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e7adb9f3-3c6d-47de-925e-43846a0f6436 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:34.891328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7adb9f3-3c6d-47de-925e-43846a0f6436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:16:34.892773 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7adb9f3-3c6d-47de-925e-43846a0f6436 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:16:34.910722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 506/1002] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:16:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:16:37.156082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14985eb0-8f78-46d8-b8f0-077bbcf3cb62 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:37.156634 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14985eb0-8f78-46d8-b8f0-077bbcf3cb62 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:37.156634 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14985eb0-8f78-46d8-b8f0-077bbcf3cb62 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:37.157979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 497/1003] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:16:37.163285 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582bc04-9794-4da0-b640-3783aa7b9ff9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:37.163668 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582bc04-9794-4da0-b640-3783aa7b9ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:37.163834 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582bc04-9794-4da0-b640-3783aa7b9ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:37.212791 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:37.231169 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d582bc04-9794-4da0-b640-3783aa7b9ff9 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:16:37.232390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d582bc04-9794-4da0-b640-3783aa7b9ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:37.336664 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 507/1004] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:16:37.621813 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14985eb0-8f78-46d8-b8f0-077bbcf3cb62 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:16:37.624123 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-14985eb0-8f78-46d8-b8f0-077bbcf3cb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:37.726061 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14985eb0-8f78-46d8-b8f0-077bbcf3cb62 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:16:37.805983 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52e50e-bc7e-4d8c-bd39-13db6f12233c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:16:37.806605 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e50e-bc7e-4d8c-bd39-13db6f12233c service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:37.806888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e50e-bc7e-4d8c-bd39-13db6f12233c service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:37.807193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e50e-bc7e-4d8c-bd39-13db6f12233c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:37.808796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ea52e50e-bc7e-4d8c-bd39-13db6f12233c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:37.826395 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:37.920849 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52e50e-bc7e-4d8c-bd39-13db6f12233c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:16:37.922215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52e50e-bc7e-4d8c-bd39-13db6f12233c service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:16:37.940894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 498/1005] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:16:40.156620 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf59d5a-1c76-45dc-9f12-91426beee385 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:40.157142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf59d5a-1c76-45dc-9f12-91426beee385 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:40.157142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf59d5a-1c76-45dc-9f12-91426beee385 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:40.158505 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 508/1006] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:16:40.163672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7509ef3-9b23-475e-9451-347924f91869 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:40.164077 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7509ef3-9b23-475e-9451-347924f91869 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:40.164222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7509ef3-9b23-475e-9451-347924f91869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:40.228965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d7509ef3-9b23-475e-9451-347924f91869 None None] MFA Rules not processed for user `e989ab80fa194e5ca4a4e53f88ae8ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:16:40.230085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d7509ef3-9b23-475e-9451-347924f91869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:40.295264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:40.327127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 499/1007] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:16:40 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:16:40.616911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf59d5a-1c76-45dc-9f12-91426beee385 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:16:40.618522 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7cf59d5a-1c76-45dc-9f12-91426beee385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:40.722639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf59d5a-1c76-45dc-9f12-91426beee385 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:16:40.746978 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:40.802714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa69a3-3166-491e-9306-5290cda09732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:16:40.803349 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa69a3-3166-491e-9306-5290cda09732 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:40.803625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa69a3-3166-491e-9306-5290cda09732 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:40.803894 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa69a3-3166-491e-9306-5290cda09732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:40.805463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a2fa69a3-3166-491e-9306-5290cda09732 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:40.914741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa69a3-3166-491e-9306-5290cda09732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:16:40.916141 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa69a3-3166-491e-9306-5290cda09732 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:16:40.933896 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 509/1008] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:16:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:16:43.340076 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de0136-f1a2-4f5f-bb4c-e5afdc961ee0 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:43.340646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de0136-f1a2-4f5f-bb4c-e5afdc961ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:43.340646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de0136-f1a2-4f5f-bb4c-e5afdc961ee0 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:43.342061 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 500/1009] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:16:43.347235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb018204-e7f0-495c-a5e8-90d782a3103c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:43.347657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb018204-e7f0-495c-a5e8-90d782a3103c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:43.347777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb018204-e7f0-495c-a5e8-90d782a3103c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:43.413921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:43.417457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-eb018204-e7f0-495c-a5e8-90d782a3103c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:16:43.419532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-eb018204-e7f0-495c-a5e8-90d782a3103c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:43.517932 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 510/1010] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:16:43.799449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de0136-f1a2-4f5f-bb4c-e5afdc961ee0 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:16:43.801018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f6de0136-f1a2-4f5f-bb4c-e5afdc961ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:43.903205 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de0136-f1a2-4f5f-bb4c-e5afdc961ee0 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:16:43.977257 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ea3b0-2511-4ecd-9c58-83c0289cc353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:16:43.977911 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ea3b0-2511-4ecd-9c58-83c0289cc353 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:43.978138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ea3b0-2511-4ecd-9c58-83c0289cc353 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:43.978404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ea3b0-2511-4ecd-9c58-83c0289cc353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:43.980019 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a14ea3b0-2511-4ecd-9c58-83c0289cc353 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:44.007615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:44.089470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ea3b0-2511-4ecd-9c58-83c0289cc353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:16:44.091596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ea3b0-2511-4ecd-9c58-83c0289cc353 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:16:44.109311 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 501/1011] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:16:46.300652 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac65b6-d9aa-4f65-993a-8cb931d1e6c7 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:46.301207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac65b6-d9aa-4f65-993a-8cb931d1e6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:46.301207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac65b6-d9aa-4f65-993a-8cb931d1e6c7 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:46.302554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 511/1012] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:16:46.307865 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994178b6-1781-4ab1-8660-f1c611791595 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:46.308304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994178b6-1781-4ab1-8660-f1c611791595 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:46.308412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994178b6-1781-4ab1-8660-f1c611791595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:46.373980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-994178b6-1781-4ab1-8660-f1c611791595 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:16:46.375216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-994178b6-1781-4ab1-8660-f1c611791595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:46.451104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:46.473410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 502/1013] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:16:46.766702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac65b6-d9aa-4f65-993a-8cb931d1e6c7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:16:46.768273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d6ac65b6-d9aa-4f65-993a-8cb931d1e6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:46.868597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac65b6-d9aa-4f65-993a-8cb931d1e6c7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:16:46.897367 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:46.942080 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1016f4-fc93-444d-9498-149b7f409489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:16:46.942686 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1016f4-fc93-444d-9498-149b7f409489 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:46.942966 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1016f4-fc93-444d-9498-149b7f409489 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:46.943269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1016f4-fc93-444d-9498-149b7f409489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:46.944850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-be1016f4-fc93-444d-9498-149b7f409489 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:47.051053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1016f4-fc93-444d-9498-149b7f409489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:16:47.052510 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1016f4-fc93-444d-9498-149b7f409489 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:16:47.070470 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 512/1014] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:16:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:16:49.516487 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bef78-28e3-4053-8a2a-7301ce8f7581 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:49.517031 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bef78-28e3-4053-8a2a-7301ce8f7581 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:49.517031 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bef78-28e3-4053-8a2a-7301ce8f7581 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:49.518354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 503/1015] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:16:49.523502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ec534-dd77-403e-bfe4-ada30c25b13b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:49.523887 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ec534-dd77-403e-bfe4-ada30c25b13b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:49.524061 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ec534-dd77-403e-bfe4-ada30c25b13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:49.590784 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-903ec534-dd77-403e-bfe4-ada30c25b13b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:16:49.591932 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-903ec534-dd77-403e-bfe4-ada30c25b13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:49.600149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:49.688247 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 513/1016] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:16:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:16:49.967686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bef78-28e3-4053-8a2a-7301ce8f7581 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:16:49.969211 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-853bef78-28e3-4053-8a2a-7301ce8f7581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:50.074155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bef78-28e3-4053-8a2a-7301ce8f7581 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:16:50.150545 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae812c-9e93-4a99-9e82-1293248382e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:16:50.151227 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae812c-9e93-4a99-9e82-1293248382e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:50.151502 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae812c-9e93-4a99-9e82-1293248382e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:50.151770 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae812c-9e93-4a99-9e82-1293248382e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:50.154065 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b7ae812c-9e93-4a99-9e82-1293248382e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:50.188297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:50.260155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae812c-9e93-4a99-9e82-1293248382e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:16:50.261577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae812c-9e93-4a99-9e82-1293248382e1 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:16:50.280625 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 504/1017] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:16:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:16:54.369399 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d711c5e-f080-4ec0-97a8-de30ca64cc76 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:54.369951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d711c5e-f080-4ec0-97a8-de30ca64cc76 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:54.369951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d711c5e-f080-4ec0-97a8-de30ca64cc76 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:54.371970 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 514/1018] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:16:54.377573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc49e33-1bbc-42e8-ad70-c2f9029cf3f9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:54.377968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc49e33-1bbc-42e8-ad70-c2f9029cf3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:54.378231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc49e33-1bbc-42e8-ad70-c2f9029cf3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:54.446176 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-0fc49e33-1bbc-42e8-ad70-c2f9029cf3f9 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:16:54.447782 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0fc49e33-1bbc-42e8-ad70-c2f9029cf3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:54.536304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:54.548741 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 505/1019] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:16:54.828541 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d711c5e-f080-4ec0-97a8-de30ca64cc76 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:16:54.830093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1d711c5e-f080-4ec0-97a8-de30ca64cc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:54.932993 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d711c5e-f080-4ec0-97a8-de30ca64cc76 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:16:54.970777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:55.006641 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f54e60-1e35-414e-ac53-d99823d0d373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:16:55.007302 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f54e60-1e35-414e-ac53-d99823d0d373 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:55.007577 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f54e60-1e35-414e-ac53-d99823d0d373 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:55.007845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f54e60-1e35-414e-ac53-d99823d0d373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:55.009406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-83f54e60-1e35-414e-ac53-d99823d0d373 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:55.115639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f54e60-1e35-414e-ac53-d99823d0d373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:16:55.117072 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f54e60-1e35-414e-ac53-d99823d0d373 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:16:55.135181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 515/1020] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:16:57.847639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd68374-cf9b-4507-93f3-2b965df73ed8 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:57.848134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd68374-cf9b-4507-93f3-2b965df73ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:57.848134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd68374-cf9b-4507-93f3-2b965df73ed8 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:57.849432 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 506/1021] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:16:57.854677 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840374fe-3f73-43e1-9d9d-5bab4840a90f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:57.855123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840374fe-3f73-43e1-9d9d-5bab4840a90f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:57.855824 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840374fe-3f73-43e1-9d9d-5bab4840a90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:57.922048 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-840374fe-3f73-43e1-9d9d-5bab4840a90f None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:16:57.923271 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-840374fe-3f73-43e1-9d9d-5bab4840a90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:57.941159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:58.023130 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 516/1022] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:16:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:16:58.317084 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd68374-cf9b-4507-93f3-2b965df73ed8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:16:58.318609 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cdd68374-cf9b-4507-93f3-2b965df73ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:58.425553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd68374-cf9b-4507-93f3-2b965df73ed8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:16:58.514726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c497e-4328-41b2-9a2b-cd151219cbe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:16:58.515385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c497e-4328-41b2-9a2b-cd151219cbe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:16:58.515712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c497e-4328-41b2-9a2b-cd151219cbe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:16:58.515969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c497e-4328-41b2-9a2b-cd151219cbe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:16:58.517636 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5b6c497e-4328-41b2-9a2b-cd151219cbe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:16:58.562397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:16:58.630037 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c497e-4328-41b2-9a2b-cd151219cbe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:16:58.631517 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c497e-4328-41b2-9a2b-cd151219cbe3 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:16:58.650056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 507/1023] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:16:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:02.760978 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f92a1a-2bd1-474a-929f-5676689ebe15 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:02.762028 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f92a1a-2bd1-474a-929f-5676689ebe15 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:02.762112 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f92a1a-2bd1-474a-929f-5676689ebe15 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:02.763504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 517/1024] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:02.769074 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960b5ae-c8dd-4779-b02c-c61df7395bd7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:02.769510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960b5ae-c8dd-4779-b02c-c61df7395bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:02.769635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960b5ae-c8dd-4779-b02c-c61df7395bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:02.835366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b960b5ae-c8dd-4779-b02c-c61df7395bd7 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:02.836596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b960b5ae-c8dd-4779-b02c-c61df7395bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:02.937226 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:02.940214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 508/1025] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:03.221635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f92a1a-2bd1-474a-929f-5676689ebe15 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:03.223189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-97f92a1a-2bd1-474a-929f-5676689ebe15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:03.327877 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f92a1a-2bd1-474a-929f-5676689ebe15 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:03.377818 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:03.404578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9089c2c-4e8f-424e-afbe-8dd9bd75dcb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:03.405194 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9089c2c-4e8f-424e-afbe-8dd9bd75dcb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:03.405470 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9089c2c-4e8f-424e-afbe-8dd9bd75dcb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:03.405740 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9089c2c-4e8f-424e-afbe-8dd9bd75dcb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:03.407355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f9089c2c-4e8f-424e-afbe-8dd9bd75dcb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:03.519314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9089c2c-4e8f-424e-afbe-8dd9bd75dcb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:03.520764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9089c2c-4e8f-424e-afbe-8dd9bd75dcb4 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:03.540285 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 518/1026] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:08.359912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17b254-4b28-47db-81b7-9a725122adad None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:08.360326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17b254-4b28-47db-81b7-9a725122adad None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:08.360464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17b254-4b28-47db-81b7-9a725122adad None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:08.362360 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 509/1027] 10.222.0.18 () {54 vars in 921 bytes} [Fri Oct 14 02:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:08.371481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acf830-19e3-4029-ba2f-3c9fa9aba1fa None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:08.371481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acf830-19e3-4029-ba2f-3c9fa9aba1fa None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:08.371481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acf830-19e3-4029-ba2f-3c9fa9aba1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:08.469513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-30acf830-19e3-4029-ba2f-3c9fa9aba1fa None None] MFA Rules not processed for user `5e935ba63cdd4256874b99a2eb0a1a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:08.470752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-30acf830-19e3-4029-ba2f-3c9fa9aba1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:08.498111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:08.649688 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 519/1028] 10.222.0.18 () {58 vars in 1018 bytes} [Fri Oct 14 02:17:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:12.713836 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f0603-5699-49e6-83c8-7961407251ea None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:12.714564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f0603-5699-49e6-83c8-7961407251ea None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:12.714564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f0603-5699-49e6-83c8-7961407251ea None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:12.716105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 510/1029] 10.222.0.18 () {54 vars in 921 bytes} [Fri Oct 14 02:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:12.721303 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a214ea-0bdd-4686-b7ae-175c5806ce77 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:12.721736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a214ea-0bdd-4686-b7ae-175c5806ce77 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:12.721855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a214ea-0bdd-4686-b7ae-175c5806ce77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:12.804551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-10a214ea-0bdd-4686-b7ae-175c5806ce77 None None] MFA Rules not processed for user `5e935ba63cdd4256874b99a2eb0a1a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:12.805759 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-10a214ea-0bdd-4686-b7ae-175c5806ce77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:12.879927 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:12.920189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 520/1030] 10.222.0.18 () {58 vars in 1018 bytes} [Fri Oct 14 02:17:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:14.364202 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066488d-baab-4453-9561-4c7e32dd7fd6 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:14.364793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066488d-baab-4453-9561-4c7e32dd7fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:14.364793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066488d-baab-4453-9561-4c7e32dd7fd6 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:14.366075 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 511/1031] 10.222.0.18 () {54 vars in 921 bytes} [Fri Oct 14 02:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:14.371202 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eb31e-7c97-41c0-ac54-10c1c13a3e3e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:14.371627 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eb31e-7c97-41c0-ac54-10c1c13a3e3e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:14.371937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eb31e-7c97-41c0-ac54-10c1c13a3e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:14.450889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-733eb31e-7c97-41c0-ac54-10c1c13a3e3e None None] MFA Rules not processed for user `5e935ba63cdd4256874b99a2eb0a1a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:14.452128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-733eb31e-7c97-41c0-ac54-10c1c13a3e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:14.561465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 521/1032] 10.222.0.18 () {58 vars in 1018 bytes} [Fri Oct 14 02:17:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:17:18.170581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8b8de-dacf-4f9c-9e9b-1681860aa397 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:18.171662 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8b8de-dacf-4f9c-9e9b-1681860aa397 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:18.172222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8b8de-dacf-4f9c-9e9b-1681860aa397 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:18.175018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 512/1033] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:18.180054 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ab5fd-63a2-4bbd-bd3c-0955bb4f8378 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:18.180524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ab5fd-63a2-4bbd-bd3c-0955bb4f8378 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:18.180806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ab5fd-63a2-4bbd-bd3c-0955bb4f8378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:18.219662 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:18.258101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-593ab5fd-63a2-4bbd-bd3c-0955bb4f8378 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:18.259807 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-593ab5fd-63a2-4bbd-bd3c-0955bb4f8378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:18.367184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 522/1034] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:18.671856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fabc68-e376-4215-82e4-51fcdd1d58bd None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:18.672794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fabc68-e376-4215-82e4-51fcdd1d58bd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:18.673415 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fabc68-e376-4215-82e4-51fcdd1d58bd None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:18.675606 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 513/1035] 10.222.0.18 () {54 vars in 961 bytes} [Fri Oct 14 02:17:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:18.685450 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9bd78-aa56-426e-b1d5-664513c13196 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:18.685820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9bd78-aa56-426e-b1d5-664513c13196 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:18.686018 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9bd78-aa56-426e-b1d5-664513c13196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:18.768698 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b5f9bd78-aa56-426e-b1d5-664513c13196 None None] MFA Rules not processed for user `de13be55b4034477baf4c029c1a79757`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:18.770036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b5f9bd78-aa56-426e-b1d5-664513c13196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:18.826655 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:18.907849 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 523/1036] 10.222.0.18 () {58 vars in 1058 bytes} [Fri Oct 14 02:17:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:18.915433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fabc68-e376-4215-82e4-51fcdd1d58bd None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:18.917646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f8fabc68-e376-4215-82e4-51fcdd1d58bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:19.073603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fabc68-e376-4215-82e4-51fcdd1d58bd None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:19.151464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:19.194120 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dc78e-f1ae-4b76-87d3-8c65872da046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:19.194730 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dc78e-f1ae-4b76-87d3-8c65872da046 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:19.194999 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dc78e-f1ae-4b76-87d3-8c65872da046 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:19.195203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dc78e-f1ae-4b76-87d3-8c65872da046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:19.196768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-695dc78e-f1ae-4b76-87d3-8c65872da046 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:19.290774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dc78e-f1ae-4b76-87d3-8c65872da046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:19.292524 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dc78e-f1ae-4b76-87d3-8c65872da046 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:19.314950 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 514/1037] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:21.704954 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d01fe1-ac58-429b-844c-6b256a1baff7 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:21.705554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d01fe1-ac58-429b-844c-6b256a1baff7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:21.705825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d01fe1-ac58-429b-844c-6b256a1baff7 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:21.707482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 524/1038] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:21.713582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c027d33-29ec-463b-962c-5468726a4801 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:21.714046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c027d33-29ec-463b-962c-5468726a4801 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:21.714201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c027d33-29ec-463b-962c-5468726a4801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:21.806373 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-8c027d33-29ec-463b-962c-5468726a4801 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:21.808906 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8c027d33-29ec-463b-962c-5468726a4801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:21.855444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:21.923881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 515/1039] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:17:22.216036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d01fe1-ac58-429b-844c-6b256a1baff7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:22.217531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-16d01fe1-ac58-429b-844c-6b256a1baff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:22.398936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d01fe1-ac58-429b-844c-6b256a1baff7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:22.412547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:22.539349 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79685aed-8052-4c78-ba56-dd3690008d9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:22.539349 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79685aed-8052-4c78-ba56-dd3690008d9b service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:22.540468 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79685aed-8052-4c78-ba56-dd3690008d9b service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:22.540468 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79685aed-8052-4c78-ba56-dd3690008d9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:22.542227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-79685aed-8052-4c78-ba56-dd3690008d9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:22.634272 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79685aed-8052-4c78-ba56-dd3690008d9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:22.636753 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79685aed-8052-4c78-ba56-dd3690008d9b service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:22.656009 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 525/1040] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:25.143465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe32a35-7057-4ab0-920f-a598b01a5922 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:25.144016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe32a35-7057-4ab0-920f-a598b01a5922 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:25.144016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe32a35-7057-4ab0-920f-a598b01a5922 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:25.145476 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 516/1041] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:25.150997 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a850de0-b724-4d2c-903a-8dda890524b5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:25.151445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a850de0-b724-4d2c-903a-8dda890524b5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:25.151612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a850de0-b724-4d2c-903a-8dda890524b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:25.167340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:25.219786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1a850de0-b724-4d2c-903a-8dda890524b5 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:25.220988 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1a850de0-b724-4d2c-903a-8dda890524b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:25.321012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 526/1042] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:17:25.612667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4debff-ade8-4e43-8406-38fd188c9589 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:25.612981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4debff-ade8-4e43-8406-38fd188c9589 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:25.613206 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4debff-ade8-4e43-8406-38fd188c9589 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:25.614744 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 517/1043] 10.222.0.18 () {54 vars in 961 bytes} [Fri Oct 14 02:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:25.620172 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4211f-2170-474a-98f9-5399d2d2c6d7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:25.620558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4211f-2170-474a-98f9-5399d2d2c6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:25.620927 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4211f-2170-474a-98f9-5399d2d2c6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:25.700452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a7a4211f-2170-474a-98f9-5399d2d2c6d7 None None] MFA Rules not processed for user `de13be55b4034477baf4c029c1a79757`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:25.701792 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a7a4211f-2170-474a-98f9-5399d2d2c6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:25.744925 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:25.828590 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 527/1044] 10.222.0.18 () {58 vars in 1058 bytes} [Fri Oct 14 02:17:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:25.837161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4debff-ade8-4e43-8406-38fd188c9589 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:25.839383 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0c4debff-ade8-4e43-8406-38fd188c9589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:25.992309 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4debff-ade8-4e43-8406-38fd188c9589 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:26.014341 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:26.114761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b5082-92a0-4f7a-bcaf-37c7622ac0f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:26.115402 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b5082-92a0-4f7a-bcaf-37c7622ac0f0 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:26.115677 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b5082-92a0-4f7a-bcaf-37c7622ac0f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:26.115948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b5082-92a0-4f7a-bcaf-37c7622ac0f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:26.117527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5a4b5082-92a0-4f7a-bcaf-37c7622ac0f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:26.216407 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b5082-92a0-4f7a-bcaf-37c7622ac0f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:26.219513 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b5082-92a0-4f7a-bcaf-37c7622ac0f0 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:26.239104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 518/1045] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:26.394002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd82fa2-daee-4073-8920-6761785890de None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:26.394531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd82fa2-daee-4073-8920-6761785890de None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:26.394531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd82fa2-daee-4073-8920-6761785890de None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:26.396121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 528/1046] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:26.402381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bccc2d-3e02-4456-82c6-ddf01fe3495f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:26.402709 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bccc2d-3e02-4456-82c6-ddf01fe3495f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:26.402935 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bccc2d-3e02-4456-82c6-ddf01fe3495f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:26.438633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:26.483594 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-32bccc2d-3e02-4456-82c6-ddf01fe3495f None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:26.485275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-32bccc2d-3e02-4456-82c6-ddf01fe3495f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:26.598645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 519/1047] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:17:27.004135 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98ab6a-c62a-4941-bfed-1aaf38c5987c None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:27.004416 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98ab6a-c62a-4941-bfed-1aaf38c5987c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:27.004832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98ab6a-c62a-4941-bfed-1aaf38c5987c None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:27.006815 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 529/1048] 10.222.0.18 () {54 vars in 963 bytes} [Fri Oct 14 02:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:27.014470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f31362-51ea-439d-9084-8d71c61ae7cc None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:27.014930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f31362-51ea-439d-9084-8d71c61ae7cc None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:27.015408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f31362-51ea-439d-9084-8d71c61ae7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:27.109149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-84f31362-51ea-439d-9084-8d71c61ae7cc None None] MFA Rules not processed for user `eb7a43e284e1401688b2e202c67a5eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:27.110712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-84f31362-51ea-439d-9084-8d71c61ae7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:27.189308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:27.239659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 520/1049] 10.222.0.18 () {58 vars in 1060 bytes} [Fri Oct 14 02:17:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:27.240658 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98ab6a-c62a-4941-bfed-1aaf38c5987c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:27.242251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0e98ab6a-c62a-4941-bfed-1aaf38c5987c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:27.372999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98ab6a-c62a-4941-bfed-1aaf38c5987c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:27.428318 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:27.472554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c9938-0ea7-48a4-859e-cac7c8b9228b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:27.473191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c9938-0ea7-48a4-859e-cac7c8b9228b service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:27.473501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c9938-0ea7-48a4-859e-cac7c8b9228b service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:27.473801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c9938-0ea7-48a4-859e-cac7c8b9228b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:27.475404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-996c9938-0ea7-48a4-859e-cac7c8b9228b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:27.648454 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c9938-0ea7-48a4-859e-cac7c8b9228b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:27.654655 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c9938-0ea7-48a4-859e-cac7c8b9228b service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:27.703116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 530/1050] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:28.500113 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b20fd9-51bb-44f4-bda6-05fbc483a9bb None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:28.500643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b20fd9-51bb-44f4-bda6-05fbc483a9bb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:28.500643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b20fd9-51bb-44f4-bda6-05fbc483a9bb None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:28.501999 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 521/1051] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:28.507289 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f9b31-3c33-4759-8b60-8318e57a1462 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:28.507750 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f9b31-3c33-4759-8b60-8318e57a1462 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:28.507952 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f9b31-3c33-4759-8b60-8318e57a1462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:28.573994 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-749f9b31-3c33-4759-8b60-8318e57a1462 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:28.575267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-749f9b31-3c33-4759-8b60-8318e57a1462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:28.595807 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:28.671338 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 531/1052] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:28.961593 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b20fd9-51bb-44f4-bda6-05fbc483a9bb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:28.963788 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-19b20fd9-51bb-44f4-bda6-05fbc483a9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:29.110924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b20fd9-51bb-44f4-bda6-05fbc483a9bb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:29.163157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:29.238937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9e8be-ca91-4531-b22e-b8f49e793162 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:29.239890 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9e8be-ca91-4531-b22e-b8f49e793162 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:29.240173 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9e8be-ca91-4531-b22e-b8f49e793162 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:29.240446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9e8be-ca91-4531-b22e-b8f49e793162 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:29.242148 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dba9e8be-ca91-4531-b22e-b8f49e793162 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:29.340669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9e8be-ca91-4531-b22e-b8f49e793162 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:29.342317 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9e8be-ca91-4531-b22e-b8f49e793162 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:29.362029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 522/1053] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:31.566211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e9cbc-1e81-48d4-bd43-d194b7fce494 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:31.566772 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e9cbc-1e81-48d4-bd43-d194b7fce494 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:31.566772 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e9cbc-1e81-48d4-bd43-d194b7fce494 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:31.568737 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 532/1054] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:31.574933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d513a6a-c1e0-4f41-a4dc-c3fc8b94f204 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:31.575441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d513a6a-c1e0-4f41-a4dc-c3fc8b94f204 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:31.575521 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d513a6a-c1e0-4f41-a4dc-c3fc8b94f204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:31.646159 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5d513a6a-c1e0-4f41-a4dc-c3fc8b94f204 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:31.647388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5d513a6a-c1e0-4f41-a4dc-c3fc8b94f204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:31.655473 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:31.764522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 523/1055] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:32.073227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e9cbc-1e81-48d4-bd43-d194b7fce494 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:32.075053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-086e9cbc-1e81-48d4-bd43-d194b7fce494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:32.229265 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e9cbc-1e81-48d4-bd43-d194b7fce494 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:32.233210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:32.359134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b8dd7-b169-48ef-8286-f3f1d2aa7558 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:32.359651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b8dd7-b169-48ef-8286-f3f1d2aa7558 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:32.359926 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b8dd7-b169-48ef-8286-f3f1d2aa7558 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:32.360192 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b8dd7-b169-48ef-8286-f3f1d2aa7558 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:32.362038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f11b8dd7-b169-48ef-8286-f3f1d2aa7558 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:32.459182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b8dd7-b169-48ef-8286-f3f1d2aa7558 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:32.461307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b8dd7-b169-48ef-8286-f3f1d2aa7558 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:32.483361 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 533/1056] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:34.690619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263fc11-9c3c-4e79-a403-79b33faf580f None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:34.691212 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263fc11-9c3c-4e79-a403-79b33faf580f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:34.691212 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263fc11-9c3c-4e79-a403-79b33faf580f None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:34.692693 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 524/1057] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:34.697719 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b713d-40e4-48cc-a6e8-c77a5b6a32c4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:34.698184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b713d-40e4-48cc-a6e8-c77a5b6a32c4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:34.698337 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b713d-40e4-48cc-a6e8-c77a5b6a32c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:34.709786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:34.767832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1e0b713d-40e4-48cc-a6e8-c77a5b6a32c4 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:34.769025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1e0b713d-40e4-48cc-a6e8-c77a5b6a32c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:34.871412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 534/1058] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:35.168307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263fc11-9c3c-4e79-a403-79b33faf580f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:35.170296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7263fc11-9c3c-4e79-a403-79b33faf580f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:35.314377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:35.325302 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263fc11-9c3c-4e79-a403-79b33faf580f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:35.453333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98522e2-e8f4-4de6-b78b-57bb5914a6ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:35.454012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98522e2-e8f4-4de6-b78b-57bb5914a6ca service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:35.454434 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98522e2-e8f4-4de6-b78b-57bb5914a6ca service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:35.454918 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98522e2-e8f4-4de6-b78b-57bb5914a6ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:35.456953 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d98522e2-e8f4-4de6-b78b-57bb5914a6ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:35.560744 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98522e2-e8f4-4de6-b78b-57bb5914a6ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:35.562422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98522e2-e8f4-4de6-b78b-57bb5914a6ca service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:35.574583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:35.583996 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 525/1059] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:38.011533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13467f-72b0-46be-b391-38e109b6dd81 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:38.012060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13467f-72b0-46be-b391-38e109b6dd81 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:38.012155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13467f-72b0-46be-b391-38e109b6dd81 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:38.013452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 535/1060] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:38.018760 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395054d-75f8-47b8-bae2-0637c171e5c2 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:38.019186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395054d-75f8-47b8-bae2-0637c171e5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:38.019348 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395054d-75f8-47b8-bae2-0637c171e5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:38.090258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5395054d-75f8-47b8-bae2-0637c171e5c2 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:38.091539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5395054d-75f8-47b8-bae2-0637c171e5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:38.193928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 526/1061] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:38.480404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13467f-72b0-46be-b391-38e109b6dd81 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:38.481918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bb13467f-72b0-46be-b391-38e109b6dd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:38.570650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:38.635900 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13467f-72b0-46be-b391-38e109b6dd81 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:38.756389 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8320e4-a675-4a5f-84f6-f7f8f448ea7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:38.757019 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8320e4-a675-4a5f-84f6-f7f8f448ea7e service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:38.757297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8320e4-a675-4a5f-84f6-f7f8f448ea7e service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:38.757568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8320e4-a675-4a5f-84f6-f7f8f448ea7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:38.759201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ee8320e4-a675-4a5f-84f6-f7f8f448ea7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:38.809961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:38.852629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8320e4-a675-4a5f-84f6-f7f8f448ea7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:38.854471 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8320e4-a675-4a5f-84f6-f7f8f448ea7e service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:38.874491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 536/1062] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:41.026433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2038ee9-77cb-4091-bf6d-e5230709abff None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:41.027158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2038ee9-77cb-4091-bf6d-e5230709abff None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:41.027158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2038ee9-77cb-4091-bf6d-e5230709abff None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:41.028439 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 527/1063] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:41.033875 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c667b-a161-4897-9049-864b1b01bde5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:41.034310 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c667b-a161-4897-9049-864b1b01bde5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:41.034495 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c667b-a161-4897-9049-864b1b01bde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:41.100407 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b11c667b-a161-4897-9049-864b1b01bde5 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:41.101614 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b11c667b-a161-4897-9049-864b1b01bde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:41.202441 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 537/1064] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:41.494658 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2038ee9-77cb-4091-bf6d-e5230709abff None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:41.496203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a2038ee9-77cb-4091-bf6d-e5230709abff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:41.592237 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:41.676280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2038ee9-77cb-4091-bf6d-e5230709abff None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:41.804334 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de4d81-1a95-4ad5-a696-cbac0cdef0e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:41.805032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de4d81-1a95-4ad5-a696-cbac0cdef0e3 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:41.805493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de4d81-1a95-4ad5-a696-cbac0cdef0e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:41.805911 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de4d81-1a95-4ad5-a696-cbac0cdef0e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:41.807602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-12de4d81-1a95-4ad5-a696-cbac0cdef0e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:41.849886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:41.905882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de4d81-1a95-4ad5-a696-cbac0cdef0e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:41.907522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de4d81-1a95-4ad5-a696-cbac0cdef0e3 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:41.925031 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 528/1065] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:44.189987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925fa29-fd3b-4378-a2c4-00ff99cb2aa0 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:44.190703 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925fa29-fd3b-4378-a2c4-00ff99cb2aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:44.190703 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925fa29-fd3b-4378-a2c4-00ff99cb2aa0 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:44.191935 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 538/1066] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:44.197791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccebb6-9859-414c-9221-e4e98f2b69a7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:44.198235 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccebb6-9859-414c-9221-e4e98f2b69a7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:44.198340 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccebb6-9859-414c-9221-e4e98f2b69a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:44.268940 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-50ccebb6-9859-414c-9221-e4e98f2b69a7 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:44.270138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-50ccebb6-9859-414c-9221-e4e98f2b69a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:44.381951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 529/1067] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:17:44.743855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925fa29-fd3b-4378-a2c4-00ff99cb2aa0 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:44.745726 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7925fa29-fd3b-4378-a2c4-00ff99cb2aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:44.782251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:44.912349 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925fa29-fd3b-4378-a2c4-00ff99cb2aa0 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:45.019629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:45.032554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c95bc-441a-45c6-bdc0-ee3986009027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:45.033168 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c95bc-441a-45c6-bdc0-ee3986009027 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:45.033443 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c95bc-441a-45c6-bdc0-ee3986009027 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:45.033712 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c95bc-441a-45c6-bdc0-ee3986009027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:45.035357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-425c95bc-441a-45c6-bdc0-ee3986009027 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:45.125666 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c95bc-441a-45c6-bdc0-ee3986009027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:45.127300 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c95bc-441a-45c6-bdc0-ee3986009027 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:45.145891 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 539/1068] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:46.999768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bbc17-4b82-4098-9e2a-d6502be2773c None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:47.000354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bbc17-4b82-4098-9e2a-d6502be2773c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:47.000354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bbc17-4b82-4098-9e2a-d6502be2773c None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:47.001708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 530/1069] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:47.006979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae26954-a92d-4daf-ba1c-cb7b909d7319 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:47.007443 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae26954-a92d-4daf-ba1c-cb7b909d7319 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:47.007559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae26954-a92d-4daf-ba1c-cb7b909d7319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:47.075449 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-aae26954-a92d-4daf-ba1c-cb7b909d7319 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:47.076656 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-aae26954-a92d-4daf-ba1c-cb7b909d7319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:47.136315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:47.176756 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 540/1070] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:47.315345 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f168934-2f79-4325-a237-cc734d90af32 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:47.315597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f168934-2f79-4325-a237-cc734d90af32 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:47.315884 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f168934-2f79-4325-a237-cc734d90af32 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:47.317255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 531/1071] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:47.323284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779852-df48-4df1-b312-220ee94016a9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:47.323571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779852-df48-4df1-b312-220ee94016a9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:47.323838 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779852-df48-4df1-b312-220ee94016a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:47.390614 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f0779852-df48-4df1-b312-220ee94016a9 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:47.391830 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f0779852-df48-4df1-b312-220ee94016a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:47.483915 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46c8a5-ae8c-4a4a-838f-90df8731a064 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:47.484200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46c8a5-ae8c-4a4a-838f-90df8731a064 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:47.484473 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46c8a5-ae8c-4a4a-838f-90df8731a064 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:47.487052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 532/1072] 10.222.0.18 () {54 vars in 962 bytes} [Fri Oct 14 02:17:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:17:47.490405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 541/1073] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:47.493101 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc3a62-0596-4731-9b3a-b9287d72f14c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:47.493546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc3a62-0596-4731-9b3a-b9287d72f14c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:47.493825 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc3a62-0596-4731-9b3a-b9287d72f14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:47.509988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:47.571824 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-99bc3a62-0596-4731-9b3a-b9287d72f14c None None] MFA Rules not processed for user `8c05a1b303c2418fb2941b8ef56f8972`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:47.572930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-99bc3a62-0596-4731-9b3a-b9287d72f14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:47.646779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 533/1074] 10.222.0.18 () {58 vars in 1059 bytes} [Fri Oct 14 02:17:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:17:47.655835 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0779852-df48-4df1-b312-220ee94016a9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:47.657143 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f0779852-df48-4df1-b312-220ee94016a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:47.706622 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:47.753767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0779852-df48-4df1-b312-220ee94016a9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:47.790782 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc3a62-0596-4731-9b3a-b9287d72f14c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:47.792080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-99bc3a62-0596-4731-9b3a-b9287d72f14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:47.833089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ac402-ade8-4b55-a2d8-4bbcfe882942 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c05a1b303c2418fb2941b8ef56f8972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:47.833618 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ac402-ade8-4b55-a2d8-4bbcfe882942 service glance] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:47.833897 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ac402-ade8-4b55-a2d8-4bbcfe882942 service glance] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:47.834177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ac402-ade8-4b55-a2d8-4bbcfe882942 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:47.835810 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3b3ac402-ade8-4b55-a2d8-4bbcfe882942 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:47.898169 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:47.912945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc3a62-0596-4731-9b3a-b9287d72f14c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:47.946937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ac402-ade8-4b55-a2d8-4bbcfe882942 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:47.948406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ac402-ade8-4b55-a2d8-4bbcfe882942 service glance] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:47.952287 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:47.969984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 542/1075] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:48.028329 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f64f1-b597-495e-9b79-0b37a7487de0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:48.028945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f64f1-b597-495e-9b79-0b37a7487de0 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:48.029223 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f64f1-b597-495e-9b79-0b37a7487de0 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:48.029496 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f64f1-b597-495e-9b79-0b37a7487de0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:48.031111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-207f64f1-b597-495e-9b79-0b37a7487de0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:48.122198 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f64f1-b597-495e-9b79-0b37a7487de0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:48.123751 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f64f1-b597-495e-9b79-0b37a7487de0 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:48.135091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:48.144098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 534/1076] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:48.740863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662de847-7645-4755-8d20-c3a25161c46c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:48.741364 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662de847-7645-4755-8d20-c3a25161c46c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:48.741468 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662de847-7645-4755-8d20-c3a25161c46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:48.809598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-662de847-7645-4755-8d20-c3a25161c46c None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:48.811328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-662de847-7645-4755-8d20-c3a25161c46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:48.900889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 543/1077] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:17:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:48.946532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e0423-a708-4279-ac3d-da87bade728d None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:48.946815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e0423-a708-4279-ac3d-da87bade728d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:48.947112 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e0423-a708-4279-ac3d-da87bade728d None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:48.948472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 535/1078] 10.222.0.18 () {54 vars in 943 bytes} [Fri Oct 14 02:17:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:48.955089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df1e56-71c7-4fd9-aa78-ea777f512041 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:48.955400 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df1e56-71c7-4fd9-aa78-ea777f512041 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:48.955679 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df1e56-71c7-4fd9-aa78-ea777f512041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:49.033361 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b3df1e56-71c7-4fd9-aa78-ea777f512041 None None] MFA Rules not processed for user `cf0efd6d12ad458388e0c3574bbedc6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:49.034566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b3df1e56-71c7-4fd9-aa78-ea777f512041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:49.055870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:49.145506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 544/1079] 10.222.0.18 () {58 vars in 1040 bytes} [Fri Oct 14 02:17:48 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:49.155176 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e0423-a708-4279-ac3d-da87bade728d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:49.156671 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-363e0423-a708-4279-ac3d-da87bade728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:49.292708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e0423-a708-4279-ac3d-da87bade728d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:49.375192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:49.399260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754855d-db97-4ba6-9f58-32babb7785ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:49.399879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754855d-db97-4ba6-9f58-32babb7785ee service swift] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:49.400161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754855d-db97-4ba6-9f58-32babb7785ee service swift] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:49.400437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754855d-db97-4ba6-9f58-32babb7785ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:49.402157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8754855d-db97-4ba6-9f58-32babb7785ee service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:49.485904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754855d-db97-4ba6-9f58-32babb7785ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:49.487567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754855d-db97-4ba6-9f58-32babb7785ee service swift] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:49.496355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 536/1080] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:17:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:17:50.339513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d669e9-4efc-41c9-900d-72a283968413 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:50.340056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d669e9-4efc-41c9-900d-72a283968413 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:50.341532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d669e9-4efc-41c9-900d-72a283968413 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:50.342343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 545/1081] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:50.347834 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0d254-c69d-4941-9a18-fe9ad12bf40c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:50.348623 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0d254-c69d-4941-9a18-fe9ad12bf40c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:50.348899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0d254-c69d-4941-9a18-fe9ad12bf40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:50.424162 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-04f0d254-c69d-4941-9a18-fe9ad12bf40c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:50.425710 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-04f0d254-c69d-4941-9a18-fe9ad12bf40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:50.500276 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:50.539312 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 537/1082] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:50.837763 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d669e9-4efc-41c9-900d-72a283968413 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:50.839598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-10d669e9-4efc-41c9-900d-72a283968413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:50.998900 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:51.005885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d669e9-4efc-41c9-900d-72a283968413 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:51.138107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94067eaf-5900-4c1a-bd44-a13fa35de25e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:51.138817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94067eaf-5900-4c1a-bd44-a13fa35de25e service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:51.139193 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94067eaf-5900-4c1a-bd44-a13fa35de25e service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:51.139460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94067eaf-5900-4c1a-bd44-a13fa35de25e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:51.141036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-94067eaf-5900-4c1a-bd44-a13fa35de25e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:51.243722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94067eaf-5900-4c1a-bd44-a13fa35de25e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:51.245538 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94067eaf-5900-4c1a-bd44-a13fa35de25e service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:51.260617 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:51.267867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 546/1083] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:17:53.509599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf399e0-7b42-4a90-be65-cb91f221cef3 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:53.510191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf399e0-7b42-4a90-be65-cb91f221cef3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:53.510191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf399e0-7b42-4a90-be65-cb91f221cef3 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:53.511559 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 538/1084] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:17:53.516797 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0963a4-feea-4988-8163-06657106c37e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:53.517216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0963a4-feea-4988-8163-06657106c37e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:53.517402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0963a4-feea-4988-8163-06657106c37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:53.589634 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1e0963a4-feea-4988-8163-06657106c37e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:17:53.591281 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1e0963a4-feea-4988-8163-06657106c37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:53.699121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 547/1085] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:17:53.990359 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf399e0-7b42-4a90-be65-cb91f221cef3 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:17:53.992544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bcf399e0-7b42-4a90-be65-cb91f221cef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:54.161039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf399e0-7b42-4a90-be65-cb91f221cef3 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:17:54.218330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:17:54.287738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecaaeea-b874-47e9-87e5-60120fd1a14a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:17:54.288410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaaeea-b874-47e9-87e5-60120fd1a14a service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:17:54.288682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaaeea-b874-47e9-87e5-60120fd1a14a service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:17:54.288960 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaaeea-b874-47e9-87e5-60120fd1a14a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:17:54.290591 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2ecaaeea-b874-47e9-87e5-60120fd1a14a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:17:54.388972 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecaaeea-b874-47e9-87e5-60120fd1a14a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:17:54.390624 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecaaeea-b874-47e9-87e5-60120fd1a14a service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:17:54.410545 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 539/1086] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:03.697471 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=66 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:04.408017 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=74 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:12.059737 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa39c3-d6fd-4636-a68e-0319523ddf0c None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:12.060278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa39c3-d6fd-4636-a68e-0319523ddf0c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:12.060278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa39c3-d6fd-4636-a68e-0319523ddf0c None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:12.061778 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 548/1087] 10.222.0.18 () {54 vars in 918 bytes} [Fri Oct 14 02:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:12.066642 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12d5ad-b5bd-45dc-9acc-9089616c67aa None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:12.067077 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12d5ad-b5bd-45dc-9acc-9089616c67aa None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:12.067251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12d5ad-b5bd-45dc-9acc-9089616c67aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:12.141581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1c12d5ad-b5bd-45dc-9acc-9089616c67aa None None] MFA Rules not processed for user `5e935ba63cdd4256874b99a2eb0a1a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:12.142775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1c12d5ad-b5bd-45dc-9acc-9089616c67aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:12.246680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 540/1088] 10.222.0.18 () {58 vars in 1015 bytes} [Fri Oct 14 02:18:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:12.260838 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451df386-085e-48c9-90d7-cb2b9df4eb16 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:12.261123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451df386-085e-48c9-90d7-cb2b9df4eb16 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:12.261409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451df386-085e-48c9-90d7-cb2b9df4eb16 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:12.262769 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 549/1089] 10.222.0.18 () {54 vars in 918 bytes} [Fri Oct 14 02:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:12.267475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba5eba-585f-45b8-86cf-b2011e63f144 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:12.267760 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba5eba-585f-45b8-86cf-b2011e63f144 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:12.268033 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba5eba-585f-45b8-86cf-b2011e63f144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:12.341210 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:12.344537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-76ba5eba-585f-45b8-86cf-b2011e63f144 None None] MFA Rules not processed for user `de13be55b4034477baf4c029c1a79757`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:12.345937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-76ba5eba-585f-45b8-86cf-b2011e63f144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:12.458525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 541/1090] 10.222.0.18 () {58 vars in 1015 bytes} [Fri Oct 14 02:18:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:15.010958 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36908734-7ac6-4211-8aee-e5b3222d0804 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:15.011476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36908734-7ac6-4211-8aee-e5b3222d0804 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:15.011581 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36908734-7ac6-4211-8aee-e5b3222d0804 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:15.012884 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 550/1091] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:15.018464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccbc85-df36-4059-bae7-7b5cf5572224 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:15.018876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccbc85-df36-4059-bae7-7b5cf5572224 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:15.019032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccbc85-df36-4059-bae7-7b5cf5572224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:15.084838 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-26ccbc85-df36-4059-bae7-7b5cf5572224 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:15.086035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-26ccbc85-df36-4059-bae7-7b5cf5572224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:15.139000 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:15.179946 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 542/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:15.466977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36908734-7ac6-4211-8aee-e5b3222d0804 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:15.468580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-36908734-7ac6-4211-8aee-e5b3222d0804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:15.616554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36908734-7ac6-4211-8aee-e5b3222d0804 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:15.688418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:15.728062 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae6be7-3559-4d33-b506-ecd53bbf9f45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:15.728670 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae6be7-3559-4d33-b506-ecd53bbf9f45 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:15.728943 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae6be7-3559-4d33-b506-ecd53bbf9f45 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:15.729211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae6be7-3559-4d33-b506-ecd53bbf9f45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:15.730784 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-07ae6be7-3559-4d33-b506-ecd53bbf9f45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:15.818908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae6be7-3559-4d33-b506-ecd53bbf9f45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:15.820512 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae6be7-3559-4d33-b506-ecd53bbf9f45 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:15.839002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 551/1093] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:16.973463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec470f-eee3-4cd0-915a-59a56152b884 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:16.974033 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec470f-eee3-4cd0-915a-59a56152b884 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:16.974033 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec470f-eee3-4cd0-915a-59a56152b884 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:16.975439 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 543/1094] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 02:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:16.981679 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6c332-5506-4c3f-9c4f-b5f773d33f9a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:16.982095 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6c332-5506-4c3f-9c4f-b5f773d33f9a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:16.982198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6c332-5506-4c3f-9c4f-b5f773d33f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:17.056359 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bef6c332-5506-4c3f-9c4f-b5f773d33f9a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:17.057534 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bef6c332-5506-4c3f-9c4f-b5f773d33f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:17.084083 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:17.151911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 552/1095] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 02:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:17.307255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec470f-eee3-4cd0-915a-59a56152b884 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:17.308817 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6eec470f-eee3-4cd0-915a-59a56152b884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:17.450614 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec470f-eee3-4cd0-915a-59a56152b884 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:17.490201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:17.564492 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf731e3-8678-4b33-b093-dd22c44783cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:17.565027 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf731e3-8678-4b33-b093-dd22c44783cf service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:17.565305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf731e3-8678-4b33-b093-dd22c44783cf service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:17.565579 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf731e3-8678-4b33-b093-dd22c44783cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:17.567214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8bf731e3-8678-4b33-b093-dd22c44783cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:17.655875 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf731e3-8678-4b33-b093-dd22c44783cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:17.657455 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf731e3-8678-4b33-b093-dd22c44783cf service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:17.675171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 544/1096] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:18.805677 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45865e56-84ba-4956-a6b0-5d1706430bfd None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:18.806296 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45865e56-84ba-4956-a6b0-5d1706430bfd None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:18.806296 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45865e56-84ba-4956-a6b0-5d1706430bfd None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:18.807604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 553/1097] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 02:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:18.814188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca106a-8e46-4c3f-a11e-44717753c81b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:18.814658 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca106a-8e46-4c3f-a11e-44717753c81b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:18.814760 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca106a-8e46-4c3f-a11e-44717753c81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:18.870235 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:18.891997 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a3ca106a-8e46-4c3f-a11e-44717753c81b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:18.893197 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a3ca106a-8e46-4c3f-a11e-44717753c81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:18.987746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 545/1098] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 02:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:19.139278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45865e56-84ba-4956-a6b0-5d1706430bfd None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:19.140790 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-45865e56-84ba-4956-a6b0-5d1706430bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:19.286332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45865e56-84ba-4956-a6b0-5d1706430bfd None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:19.298729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:19.399162 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cd78d-5d5c-4966-9230-aae749df2c87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:19.399773 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cd78d-5d5c-4966-9230-aae749df2c87 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:19.400052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cd78d-5d5c-4966-9230-aae749df2c87 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:19.400324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cd78d-5d5c-4966-9230-aae749df2c87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:19.401902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-155cd78d-5d5c-4966-9230-aae749df2c87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:19.489736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cd78d-5d5c-4966-9230-aae749df2c87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:19.491366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cd78d-5d5c-4966-9230-aae749df2c87 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:19.509417 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 554/1099] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:20.617811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c18623-f232-447e-bab8-459324b3d33c None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:20.618353 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c18623-f232-447e-bab8-459324b3d33c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:20.618353 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c18623-f232-447e-bab8-459324b3d33c None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:20.619747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 546/1100] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 02:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:20.626024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38396b4-0ac5-4b45-a14b-e0ff2b02a46b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:20.626470 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38396b4-0ac5-4b45-a14b-e0ff2b02a46b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:20.626576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38396b4-0ac5-4b45-a14b-e0ff2b02a46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:20.646640 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:20.702019 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b38396b4-0ac5-4b45-a14b-e0ff2b02a46b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:20.703270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b38396b4-0ac5-4b45-a14b-e0ff2b02a46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:20.800916 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 555/1101] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 02:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:20.951404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c18623-f232-447e-bab8-459324b3d33c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:20.952825 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-34c18623-f232-447e-bab8-459324b3d33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:21.059685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:21.096093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c18623-f232-447e-bab8-459324b3d33c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:21.207944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc598f11-c6fd-4f49-b57c-ac03b59bdc4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:21.209209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc598f11-c6fd-4f49-b57c-ac03b59bdc4a service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:21.209491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc598f11-c6fd-4f49-b57c-ac03b59bdc4a service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:21.209764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc598f11-c6fd-4f49-b57c-ac03b59bdc4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:21.211389 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cc598f11-c6fd-4f49-b57c-ac03b59bdc4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:21.286638 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:21.300302 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc598f11-c6fd-4f49-b57c-ac03b59bdc4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:21.302437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc598f11-c6fd-4f49-b57c-ac03b59bdc4a service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:21.320951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 547/1102] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:22.440371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946f884-38ef-45c4-ad9d-cfea0ea7a1d1 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:22.441047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946f884-38ef-45c4-ad9d-cfea0ea7a1d1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:22.441047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946f884-38ef-45c4-ad9d-cfea0ea7a1d1 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:22.442256 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 556/1103] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 02:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:22.448793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da481f22-d0b2-46d8-90e4-5cdc9def0c62 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:22.449301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da481f22-d0b2-46d8-90e4-5cdc9def0c62 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:22.449414 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da481f22-d0b2-46d8-90e4-5cdc9def0c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:22.524600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-da481f22-d0b2-46d8-90e4-5cdc9def0c62 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:22.525828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-da481f22-d0b2-46d8-90e4-5cdc9def0c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:22.620956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 548/1104] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 02:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:22.775092 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946f884-38ef-45c4-ad9d-cfea0ea7a1d1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:22.776675 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e946f884-38ef-45c4-ad9d-cfea0ea7a1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:22.860468 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:22.921926 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946f884-38ef-45c4-ad9d-cfea0ea7a1d1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:23.031565 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32fbb7-51ab-4ec4-add3-88128c5c7258 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:23.032170 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32fbb7-51ab-4ec4-add3-88128c5c7258 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:23.032445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32fbb7-51ab-4ec4-add3-88128c5c7258 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:23.032714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32fbb7-51ab-4ec4-add3-88128c5c7258 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:23.034284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ab32fbb7-51ab-4ec4-add3-88128c5c7258 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:23.081744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:23.124256 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32fbb7-51ab-4ec4-add3-88128c5c7258 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:23.125834 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32fbb7-51ab-4ec4-add3-88128c5c7258 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:23.144156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 557/1105] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:24.194470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b33fb-ef39-4986-b5ff-1c74a4d2f449 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:24.194963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b33fb-ef39-4986-b5ff-1c74a4d2f449 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:24.195121 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b33fb-ef39-4986-b5ff-1c74a4d2f449 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:24.196440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 549/1106] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 02:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:24.202721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba85ed-c10e-49b3-9188-727346972d78 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:24.203212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba85ed-c10e-49b3-9188-727346972d78 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:24.203314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba85ed-c10e-49b3-9188-727346972d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:24.278180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9eba85ed-c10e-49b3-9188-727346972d78 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:24.279453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9eba85ed-c10e-49b3-9188-727346972d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:24.374608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 558/1107] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 02:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:24.527600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b33fb-ef39-4986-b5ff-1c74a4d2f449 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:24.529126 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-180b33fb-ef39-4986-b5ff-1c74a4d2f449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:24.581501 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:24.677006 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b33fb-ef39-4986-b5ff-1c74a4d2f449 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:24.791761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566abe03-f77e-43dd-9e50-c801a87ea9ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:24.792678 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566abe03-f77e-43dd-9e50-c801a87ea9ab service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:24.792968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566abe03-f77e-43dd-9e50-c801a87ea9ab service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:24.793239 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566abe03-f77e-43dd-9e50-c801a87ea9ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:24.794837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-566abe03-f77e-43dd-9e50-c801a87ea9ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:24.801694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:24.883694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566abe03-f77e-43dd-9e50-c801a87ea9ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:24.885268 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566abe03-f77e-43dd-9e50-c801a87ea9ab service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:24.903369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 550/1108] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:25.948352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a55bd3-3a46-480e-9fe9-dc40ec25fb07 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:25.948908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a55bd3-3a46-480e-9fe9-dc40ec25fb07 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:25.948908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a55bd3-3a46-480e-9fe9-dc40ec25fb07 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:25.950265 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 559/1109] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 02:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:25.956717 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6988a7-1711-46a9-8f1f-a6815ba55ef7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:25.957144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6988a7-1711-46a9-8f1f-a6815ba55ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:25.957246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6988a7-1711-46a9-8f1f-a6815ba55ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:26.032946 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7c6988a7-1711-46a9-8f1f-a6815ba55ef7 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:26.034156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7c6988a7-1711-46a9-8f1f-a6815ba55ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:26.100877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:26.129173 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 551/1110] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 02:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:26.280744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a55bd3-3a46-480e-9fe9-dc40ec25fb07 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:26.282299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a9a55bd3-3a46-480e-9fe9-dc40ec25fb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:26.299353 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:26.425654 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a55bd3-3a46-480e-9fe9-dc40ec25fb07 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:26.512756 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:26.539090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e0679-8abc-4d67-989d-d9c1dd184c0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:26.539717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e0679-8abc-4d67-989d-d9c1dd184c0b service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:26.539994 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e0679-8abc-4d67-989d-d9c1dd184c0b service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:26.540257 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e0679-8abc-4d67-989d-d9c1dd184c0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:26.541907 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-400e0679-8abc-4d67-989d-d9c1dd184c0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:26.636047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e0679-8abc-4d67-989d-d9c1dd184c0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:26.637566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e0679-8abc-4d67-989d-d9c1dd184c0b service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:26.666776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 560/1111] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:27.753790 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e054d4-1487-41ad-9491-4830fe0032ca None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:27.754273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e054d4-1487-41ad-9491-4830fe0032ca None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:27.754376 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e054d4-1487-41ad-9491-4830fe0032ca None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:27.755949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 552/1112] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 02:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:27.762304 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59666fdd-3b0b-4598-9b61-406b8000a1d7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:27.762747 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59666fdd-3b0b-4598-9b61-406b8000a1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:27.762851 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59666fdd-3b0b-4598-9b61-406b8000a1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:27.837333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-59666fdd-3b0b-4598-9b61-406b8000a1d7 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:27.838532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-59666fdd-3b0b-4598-9b61-406b8000a1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:27.881711 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:27.936501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 561/1113] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 02:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:28.086473 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e054d4-1487-41ad-9491-4830fe0032ca None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:28.088007 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e5e054d4-1487-41ad-9491-4830fe0032ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:28.233103 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e054d4-1487-41ad-9491-4830fe0032ca None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:28.286667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:28.344474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a2ed1-b25a-46e5-a942-1eaf28a78524 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:28.345019 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a2ed1-b25a-46e5-a942-1eaf28a78524 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:28.345294 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a2ed1-b25a-46e5-a942-1eaf28a78524 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:28.345562 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a2ed1-b25a-46e5-a942-1eaf28a78524 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:28.347209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3d7a2ed1-b25a-46e5-a942-1eaf28a78524 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:28.434215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a2ed1-b25a-46e5-a942-1eaf28a78524 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:28.435814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a2ed1-b25a-46e5-a942-1eaf28a78524 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:28.454333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 553/1114] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:29.502118 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416628b-53e5-46a1-bdb3-eb29202c686e None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:29.502674 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416628b-53e5-46a1-bdb3-eb29202c686e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:29.502674 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416628b-53e5-46a1-bdb3-eb29202c686e None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:29.504014 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 562/1115] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 02:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:29.510689 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1edcc-2234-4607-a7e3-035df7abb490 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:29.511173 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1edcc-2234-4607-a7e3-035df7abb490 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:29.511286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1edcc-2234-4607-a7e3-035df7abb490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:29.585569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c8b1edcc-2234-4607-a7e3-035df7abb490 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:29.586765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c8b1edcc-2234-4607-a7e3-035df7abb490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:29.588829 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:29.681908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 554/1116] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 02:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:29.833347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416628b-53e5-46a1-bdb3-eb29202c686e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:29.834888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3416628b-53e5-46a1-bdb3-eb29202c686e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:29.981212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416628b-53e5-46a1-bdb3-eb29202c686e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:30.008884 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:30.093180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81e2c1-030e-46a9-a373-8f935e3d0574 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:30.093780 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81e2c1-030e-46a9-a373-8f935e3d0574 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:30.094055 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81e2c1-030e-46a9-a373-8f935e3d0574 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:30.094325 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81e2c1-030e-46a9-a373-8f935e3d0574 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:30.095937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ba81e2c1-030e-46a9-a373-8f935e3d0574 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:30.182884 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81e2c1-030e-46a9-a373-8f935e3d0574 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:30.184480 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81e2c1-030e-46a9-a373-8f935e3d0574 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:30.202158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 563/1117] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:32.267843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8951a67-13f8-472b-8dad-7ed6ddbd12f1 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:32.268426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8951a67-13f8-472b-8dad-7ed6ddbd12f1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:32.268426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8951a67-13f8-472b-8dad-7ed6ddbd12f1 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:32.269729 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 555/1118] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:32.275242 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bcd24-1198-463a-b0a2-99245d834222 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:32.275671 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bcd24-1198-463a-b0a2-99245d834222 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:32.275779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bcd24-1198-463a-b0a2-99245d834222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:32.316676 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:32.341569 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f33bcd24-1198-463a-b0a2-99245d834222 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:32.342765 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f33bcd24-1198-463a-b0a2-99245d834222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:32.438304 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 564/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:32.724255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8951a67-13f8-472b-8dad-7ed6ddbd12f1 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:32.725843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f8951a67-13f8-472b-8dad-7ed6ddbd12f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:32.829739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8951a67-13f8-472b-8dad-7ed6ddbd12f1 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:32.862711 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:32.904196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b002b3c-6be6-4102-bb35-613a8e8fb001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:32.904800 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b002b3c-6be6-4102-bb35-613a8e8fb001 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:32.905073 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b002b3c-6be6-4102-bb35-613a8e8fb001 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:32.905340 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b002b3c-6be6-4102-bb35-613a8e8fb001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:32.906915 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2b002b3c-6be6-4102-bb35-613a8e8fb001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:33.014455 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b002b3c-6be6-4102-bb35-613a8e8fb001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:33.015901 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b002b3c-6be6-4102-bb35-613a8e8fb001 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:33.033898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 556/1120] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:35.437659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80d65a-e04e-4fa9-a582-abf94e113af3 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:35.438204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80d65a-e04e-4fa9-a582-abf94e113af3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:35.438204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80d65a-e04e-4fa9-a582-abf94e113af3 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:35.441262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 565/1121] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:18:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:35.446776 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1b18a-3349-4ef7-9a7c-b5651366706b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:35.447317 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1b18a-3349-4ef7-9a7c-b5651366706b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:35.447435 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1b18a-3349-4ef7-9a7c-b5651366706b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:35.513265 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-93c1b18a-3349-4ef7-9a7c-b5651366706b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:35.514463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-93c1b18a-3349-4ef7-9a7c-b5651366706b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:35.527247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:35.611782 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 557/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:35.898914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80d65a-e04e-4fa9-a582-abf94e113af3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:35.900494 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ba80d65a-e04e-4fa9-a582-abf94e113af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:36.002951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80d65a-e04e-4fa9-a582-abf94e113af3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:36.015555 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:36.076354 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c8d34-fa24-4d50-98e5-3f3f18d25306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:36.076962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c8d34-fa24-4d50-98e5-3f3f18d25306 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:36.077249 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c8d34-fa24-4d50-98e5-3f3f18d25306 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:36.077516 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c8d34-fa24-4d50-98e5-3f3f18d25306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:36.079133 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c67c8d34-fa24-4d50-98e5-3f3f18d25306 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:36.187115 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c8d34-fa24-4d50-98e5-3f3f18d25306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:36.188545 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c8d34-fa24-4d50-98e5-3f3f18d25306 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:36.206299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 566/1123] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:38.620844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed707fb6-edd8-4b77-bea5-46c8c8d30fc7 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:38.621410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed707fb6-edd8-4b77-bea5-46c8c8d30fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:38.621410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed707fb6-edd8-4b77-bea5-46c8c8d30fc7 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:38.622722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 558/1124] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:38.628033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e19e1-eb97-47ba-b721-e3629b2ea1bb None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:38.628456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e19e1-eb97-47ba-b721-e3629b2ea1bb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:38.628585 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e19e1-eb97-47ba-b721-e3629b2ea1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:38.676373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:38.694251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-7c5e19e1-eb97-47ba-b721-e3629b2ea1bb None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:38.695480 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7c5e19e1-eb97-47ba-b721-e3629b2ea1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:38.792692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 567/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:39.076517 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed707fb6-edd8-4b77-bea5-46c8c8d30fc7 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:39.078087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ed707fb6-edd8-4b77-bea5-46c8c8d30fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:39.180928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed707fb6-edd8-4b77-bea5-46c8c8d30fc7 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:39.221635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:39.255746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c184a91-b830-4b0f-a958-828d42ecec43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:39.256356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c184a91-b830-4b0f-a958-828d42ecec43 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:39.256635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c184a91-b830-4b0f-a958-828d42ecec43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:39.256906 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c184a91-b830-4b0f-a958-828d42ecec43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:39.258489 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7c184a91-b830-4b0f-a958-828d42ecec43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:39.367546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c184a91-b830-4b0f-a958-828d42ecec43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:39.368906 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c184a91-b830-4b0f-a958-828d42ecec43 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:39.386739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 559/1126] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:41.667726 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008d4d4-665c-454c-8454-64fc1b8418a2 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:41.668258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008d4d4-665c-454c-8454-64fc1b8418a2 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:41.668258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008d4d4-665c-454c-8454-64fc1b8418a2 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:41.669584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 568/1127] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:41.674806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d58e63-8261-4638-804f-2fc9fdd06a1c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:41.675237 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d58e63-8261-4638-804f-2fc9fdd06a1c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:41.675380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d58e63-8261-4638-804f-2fc9fdd06a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:41.741766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a8d58e63-8261-4638-804f-2fc9fdd06a1c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:41.742977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a8d58e63-8261-4638-804f-2fc9fdd06a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:41.763091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:41.838675 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 560/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:42.124274 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008d4d4-665c-454c-8454-64fc1b8418a2 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:42.125817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c008d4d4-665c-454c-8454-64fc1b8418a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:42.228284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008d4d4-665c-454c-8454-64fc1b8418a2 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:42.248095 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:42.300645 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bc2a0-3793-444e-8f61-d04c98bd7049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:42.301759 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bc2a0-3793-444e-8f61-d04c98bd7049 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:42.302112 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bc2a0-3793-444e-8f61-d04c98bd7049 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:42.302388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bc2a0-3793-444e-8f61-d04c98bd7049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:42.304006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-446bc2a0-3793-444e-8f61-d04c98bd7049 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:42.410122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446bc2a0-3793-444e-8f61-d04c98bd7049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:42.411646 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446bc2a0-3793-444e-8f61-d04c98bd7049 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:42.429863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 569/1129] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:44.627954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b2e26-5c7a-4cf4-aa0e-879f980885a3 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:44.628496 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b2e26-5c7a-4cf4-aa0e-879f980885a3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:44.628496 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b2e26-5c7a-4cf4-aa0e-879f980885a3 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:44.630524 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 561/1130] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:18:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:44.636132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad6be8-d66a-4d00-8d0d-e27b369a83b4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:44.636564 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad6be8-d66a-4d00-8d0d-e27b369a83b4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:44.636668 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad6be8-d66a-4d00-8d0d-e27b369a83b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:44.698624 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:44.701777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-55ad6be8-d66a-4d00-8d0d-e27b369a83b4 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:44.703594 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-55ad6be8-d66a-4d00-8d0d-e27b369a83b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:44.800423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 570/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:45.087165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b2e26-5c7a-4cf4-aa0e-879f980885a3 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:45.088734 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e76b2e26-5c7a-4cf4-aa0e-879f980885a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:45.191860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b2e26-5c7a-4cf4-aa0e-879f980885a3 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:45.241668 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:45.265561 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1d746-5c65-4e6b-b97d-0812d8a53c28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:45.266162 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1d746-5c65-4e6b-b97d-0812d8a53c28 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:45.266436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1d746-5c65-4e6b-b97d-0812d8a53c28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:45.266703 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1d746-5c65-4e6b-b97d-0812d8a53c28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:45.268322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-33a1d746-5c65-4e6b-b97d-0812d8a53c28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:45.373339 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1d746-5c65-4e6b-b97d-0812d8a53c28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:45.374695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1d746-5c65-4e6b-b97d-0812d8a53c28 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:45.392690 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 562/1132] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:47.650138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f93abb-04f0-412a-9db3-55421030b4c6 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:47.650666 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f93abb-04f0-412a-9db3-55421030b4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:47.650666 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f93abb-04f0-412a-9db3-55421030b4c6 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:47.651957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 571/1133] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:47.657334 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937287c-6eac-4bbf-93e9-a232c7aa1098 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:47.657759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937287c-6eac-4bbf-93e9-a232c7aa1098 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:47.658053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937287c-6eac-4bbf-93e9-a232c7aa1098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:47.721896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d937287c-6eac-4bbf-93e9-a232c7aa1098 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:47.723774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d937287c-6eac-4bbf-93e9-a232c7aa1098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:47.752668 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:47.820837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 563/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:48.109810 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f93abb-04f0-412a-9db3-55421030b4c6 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:48.111450 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d8f93abb-04f0-412a-9db3-55421030b4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:48.213183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f93abb-04f0-412a-9db3-55421030b4c6 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:48.241667 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:48.286524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9baf99e-c609-4b3f-8dff-50816d1ae090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:48.287147 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baf99e-c609-4b3f-8dff-50816d1ae090 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:48.287418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baf99e-c609-4b3f-8dff-50816d1ae090 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:48.287680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baf99e-c609-4b3f-8dff-50816d1ae090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:48.289240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e9baf99e-c609-4b3f-8dff-50816d1ae090 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:48.395584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9baf99e-c609-4b3f-8dff-50816d1ae090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:48.397066 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9baf99e-c609-4b3f-8dff-50816d1ae090 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:48.414719 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 572/1135] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:50.596878 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c358da-2902-479d-941c-3870d630c158 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:50.597441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c358da-2902-479d-941c-3870d630c158 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:50.597441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c358da-2902-479d-941c-3870d630c158 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:50.598850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 564/1136] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:50.604877 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd8333-c291-468a-a9d1-c532693b1424 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:50.605308 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd8333-c291-468a-a9d1-c532693b1424 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:50.605426 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd8333-c291-468a-a9d1-c532693b1424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:50.670735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f2fd8333-c291-468a-a9d1-c532693b1424 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:50.671946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f2fd8333-c291-468a-a9d1-c532693b1424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:50.680147 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:50.769515 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 573/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:51.056701 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c358da-2902-479d-941c-3870d630c158 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:51.058247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-80c358da-2902-479d-941c-3870d630c158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:51.160634 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c358da-2902-479d-941c-3870d630c158 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:51.215199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:51.233187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5c824-d0d0-463c-8581-84a6fdfc9d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:51.233791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5c824-d0d0-463c-8581-84a6fdfc9d5b service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:51.234089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5c824-d0d0-463c-8581-84a6fdfc9d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:51.234356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5c824-d0d0-463c-8581-84a6fdfc9d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:51.236254 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-46c5c824-d0d0-463c-8581-84a6fdfc9d5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:51.343875 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5c824-d0d0-463c-8581-84a6fdfc9d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:51.345235 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5c824-d0d0-463c-8581-84a6fdfc9d5b service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:51.363558 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 565/1138] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:53.601470 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dbfc9-ca1d-46e2-be5c-c7d5ceb75fde None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:53.602065 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dbfc9-ca1d-46e2-be5c-c7d5ceb75fde None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:53.602065 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dbfc9-ca1d-46e2-be5c-c7d5ceb75fde None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:53.603393 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 574/1139] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:53.608624 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e305b3-e53c-40f1-8ba1-3b588953c6a1 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:53.609060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e305b3-e53c-40f1-8ba1-3b588953c6a1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:53.609163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e305b3-e53c-40f1-8ba1-3b588953c6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:53.673623 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d9e305b3-e53c-40f1-8ba1-3b588953c6a1 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:53.674735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d9e305b3-e53c-40f1-8ba1-3b588953c6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:53.713122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:53.775089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 566/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:18:54.057529 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3dbfc9-ca1d-46e2-be5c-c7d5ceb75fde None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:54.059118 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7e3dbfc9-ca1d-46e2-be5c-c7d5ceb75fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:54.163452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3dbfc9-ca1d-46e2-be5c-c7d5ceb75fde None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:54.201305 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:54.237496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c17d6-b0f9-4427-a839-e3beadc3088d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:54.238105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c17d6-b0f9-4427-a839-e3beadc3088d service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:54.238383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c17d6-b0f9-4427-a839-e3beadc3088d service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:54.238655 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c17d6-b0f9-4427-a839-e3beadc3088d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:54.240897 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-799c17d6-b0f9-4427-a839-e3beadc3088d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:54.346224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c17d6-b0f9-4427-a839-e3beadc3088d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:54.347598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c17d6-b0f9-4427-a839-e3beadc3088d service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:54.365679 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 575/1141] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:56.561982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efa9ee-979e-4b05-969c-9d2521bd6771 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:56.562571 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efa9ee-979e-4b05-969c-9d2521bd6771 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:56.562571 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efa9ee-979e-4b05-969c-9d2521bd6771 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:56.563957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 567/1142] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:56.569591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0883e-8f47-46d0-80e7-00d1a09f251c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:56.569994 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0883e-8f47-46d0-80e7-00d1a09f251c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:56.570232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0883e-8f47-46d0-80e7-00d1a09f251c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:56.638793 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-98e0883e-8f47-46d0-80e7-00d1a09f251c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:56.640048 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-98e0883e-8f47-46d0-80e7-00d1a09f251c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:56.658431 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:56.740710 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 576/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:57.023294 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efa9ee-979e-4b05-969c-9d2521bd6771 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:57.024832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-32efa9ee-979e-4b05-969c-9d2521bd6771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:57.128030 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efa9ee-979e-4b05-969c-9d2521bd6771 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:18:57.195493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:57.202511 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efbce7-e900-47c8-804f-cd6d3d24e40b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:18:57.203141 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efbce7-e900-47c8-804f-cd6d3d24e40b service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:57.203418 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efbce7-e900-47c8-804f-cd6d3d24e40b service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:57.203685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efbce7-e900-47c8-804f-cd6d3d24e40b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:57.205290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f5efbce7-e900-47c8-804f-cd6d3d24e40b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:57.312844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efbce7-e900-47c8-804f-cd6d3d24e40b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:18:57.314203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efbce7-e900-47c8-804f-cd6d3d24e40b service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:18:57.332315 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 568/1144] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:18:59.486140 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f0a49-162d-4354-9e0f-c8cbd9e6ddcf None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:59.486669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f0a49-162d-4354-9e0f-c8cbd9e6ddcf None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:59.486669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f0a49-162d-4354-9e0f-c8cbd9e6ddcf None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:59.488068 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 577/1145] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:18:59.493417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a89dca-f320-4a20-abb9-3641990fa8c6 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:18:59.493866 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a89dca-f320-4a20-abb9-3641990fa8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:18:59.493969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a89dca-f320-4a20-abb9-3641990fa8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:18:59.558371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a2a89dca-f320-4a20-abb9-3641990fa8c6 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:18:59.559584 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a2a89dca-f320-4a20-abb9-3641990fa8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:18:59.606537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:18:59.654640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 569/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:18:59.936567 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f0a49-162d-4354-9e0f-c8cbd9e6ddcf None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:18:59.938085 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-161f0a49-162d-4354-9e0f-c8cbd9e6ddcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:00.040072 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f0a49-162d-4354-9e0f-c8cbd9e6ddcf None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:19:00.087041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:00.113377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8924e-71b6-4040-ae36-27f19514505a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:19:00.113987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8924e-71b6-4040-ae36-27f19514505a service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:00.114265 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8924e-71b6-4040-ae36-27f19514505a service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:00.114549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8924e-71b6-4040-ae36-27f19514505a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:00.116203 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a2c8924e-71b6-4040-ae36-27f19514505a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:00.222865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8924e-71b6-4040-ae36-27f19514505a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:19:00.224593 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8924e-71b6-4040-ae36-27f19514505a service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:19:00.242954 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 578/1147] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:19:09.653175 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=22 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:10.240657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=66 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:20.222981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210e271-af43-44ee-b8fd-5faa3bcda1b2 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:20.223560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210e271-af43-44ee-b8fd-5faa3bcda1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:20.223560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210e271-af43-44ee-b8fd-5faa3bcda1b2 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:20.225421 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 570/1148] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:19:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:19:20.230788 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216213a-836b-4cf6-929a-9d28d65c2330 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:20.231243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216213a-836b-4cf6-929a-9d28d65c2330 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:20.231374 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216213a-836b-4cf6-929a-9d28d65c2330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:20.296840 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9216213a-836b-4cf6-929a-9d28d65c2330 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:19:20.298042 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9216213a-836b-4cf6-929a-9d28d65c2330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:20.392088 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 579/1149] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:19:20.686611 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2d2c2-5e1b-454e-937e-d738d18287f6 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:20.686895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2d2c2-5e1b-454e-937e-d738d18287f6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:20.687197 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2d2c2-5e1b-454e-937e-d738d18287f6 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:20.688583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 571/1150] 10.222.0.18 () {54 vars in 962 bytes} [Fri Oct 14 02:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:19:20.694242 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79d621-400e-43c7-95f9-84c4ab3992f9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:20.694517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79d621-400e-43c7-95f9-84c4ab3992f9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:20.694788 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79d621-400e-43c7-95f9-84c4ab3992f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:20.773025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5f79d621-400e-43c7-95f9-84c4ab3992f9 None None] MFA Rules not processed for user `8c05a1b303c2418fb2941b8ef56f8972`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:19:20.774218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5f79d621-400e-43c7-95f9-84c4ab3992f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:20.794460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:20.848823 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 580/1151] 10.222.0.18 () {58 vars in 1059 bytes} [Fri Oct 14 02:19:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:19:20.857309 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2d2c2-5e1b-454e-937e-d738d18287f6 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:19:20.858828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3df2d2c2-5e1b-454e-937e-d738d18287f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:20.960555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2d2c2-5e1b-454e-937e-d738d18287f6 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:19:21.032692 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2d931-b230-4c56-8913-734d3551e906 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c05a1b303c2418fb2941b8ef56f8972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:19:21.033299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2d931-b230-4c56-8913-734d3551e906 service glance] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:21.033573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2d931-b230-4c56-8913-734d3551e906 service glance] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:21.033838 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2d931-b230-4c56-8913-734d3551e906 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:21.035447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-29d2d931-b230-4c56-8913-734d3551e906 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:21.088583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:21.143530 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2d931-b230-4c56-8913-734d3551e906 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:19:21.144963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2d931-b230-4c56-8913-734d3551e906 service glance] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:19:21.163506 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 572/1152] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:19:23.379811 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae1a6e-fad6-472e-a4ee-4eb4ef89ea56 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:23.380359 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae1a6e-fad6-472e-a4ee-4eb4ef89ea56 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:23.380497 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae1a6e-fad6-472e-a4ee-4eb4ef89ea56 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:23.381753 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 581/1153] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:19:23.387207 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a64760-3843-4cfe-a7f9-859ae255543c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:23.387637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a64760-3843-4cfe-a7f9-859ae255543c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:23.387744 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a64760-3843-4cfe-a7f9-859ae255543c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:23.452160 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-30a64760-3843-4cfe-a7f9-859ae255543c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:19:23.453311 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-30a64760-3843-4cfe-a7f9-859ae255543c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:23.547527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 573/1154] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:19:23.816565 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae1a6e-fad6-472e-a4ee-4eb4ef89ea56 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:19:23.818801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bfae1a6e-fad6-472e-a4ee-4eb4ef89ea56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:23.941017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae1a6e-fad6-472e-a4ee-4eb4ef89ea56 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:19:23.988755 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:24.032835 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3c687-06d3-4c92-b21a-b46b0865088f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:19:24.033439 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3c687-06d3-4c92-b21a-b46b0865088f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:24.033721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3c687-06d3-4c92-b21a-b46b0865088f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:24.033988 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3c687-06d3-4c92-b21a-b46b0865088f admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:24.034617 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3c687-06d3-4c92-b21a-b46b0865088f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:19:24.035738 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-74a3c687-06d3-4c92-b21a-b46b0865088f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:19:24.035738 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 14 02:19:24.035738 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: Oct 14 02:19:24.037048 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3c687-06d3-4c92-b21a-b46b0865088f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:19:24.055579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 582/1155] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Oct 14 02:19:23 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:19:26.231640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bf956-9f7d-491d-9a79-87aacf3f17a8 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:26.232236 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bf956-9f7d-491d-9a79-87aacf3f17a8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:26.232236 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bf956-9f7d-491d-9a79-87aacf3f17a8 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:26.233544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 574/1156] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:19:26.239014 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bf65b-e1f4-4db2-bbc2-3a426ef57b9b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:26.239484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bf65b-e1f4-4db2-bbc2-3a426ef57b9b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:26.239600 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bf65b-e1f4-4db2-bbc2-3a426ef57b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:26.305549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-659bf65b-e1f4-4db2-bbc2-3a426ef57b9b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:19:26.306751 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-659bf65b-e1f4-4db2-bbc2-3a426ef57b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:26.401314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 583/1157] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:19:26.698646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bf956-9f7d-491d-9a79-87aacf3f17a8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:19:26.701102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d41bf956-9f7d-491d-9a79-87aacf3f17a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:26.711893 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:26.856582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bf956-9f7d-491d-9a79-87aacf3f17a8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:19:26.952208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:26.992313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395c8d4-5c2e-4271-9cfd-b84b4bd0ce27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:19:26.993042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395c8d4-5c2e-4271-9cfd-b84b4bd0ce27 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:26.993430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395c8d4-5c2e-4271-9cfd-b84b4bd0ce27 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:26.993800 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395c8d4-5c2e-4271-9cfd-b84b4bd0ce27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:26.995649 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c395c8d4-5c2e-4271-9cfd-b84b4bd0ce27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:27.106260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395c8d4-5c2e-4271-9cfd-b84b4bd0ce27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:19:27.107917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395c8d4-5c2e-4271-9cfd-b84b4bd0ce27 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:19:27.127959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 575/1158] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:19:29.483251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cc5c0-8e6d-4a2c-a9ea-ff289c4984de None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:29.483793 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cc5c0-8e6d-4a2c-a9ea-ff289c4984de None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:29.483793 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cc5c0-8e6d-4a2c-a9ea-ff289c4984de None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:29.485153 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 584/1159] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:19:29.490736 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6d597-f54a-4ab2-9bf9-045a01ec32c7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:29.491208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6d597-f54a-4ab2-9bf9-045a01ec32c7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:29.491315 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6d597-f54a-4ab2-9bf9-045a01ec32c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:29.555202 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-93f6d597-f54a-4ab2-9bf9-045a01ec32c7 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:19:29.556392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-93f6d597-f54a-4ab2-9bf9-045a01ec32c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:29.601921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:29.650668 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 576/1160] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:19:29.938823 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cc5c0-8e6d-4a2c-a9ea-ff289c4984de None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:19:29.940392 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-540cc5c0-8e6d-4a2c-a9ea-ff289c4984de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:29.966172 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:30.045536 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cc5c0-8e6d-4a2c-a9ea-ff289c4984de None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:19:30.117470 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518a2bb-2db9-4477-9c9a-290fda5d5e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:19:30.118079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518a2bb-2db9-4477-9c9a-290fda5d5e04 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:30.118361 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518a2bb-2db9-4477-9c9a-290fda5d5e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:30.118635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518a2bb-2db9-4477-9c9a-290fda5d5e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:30.120268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9518a2bb-2db9-4477-9c9a-290fda5d5e04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:30.194315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:30.228340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518a2bb-2db9-4477-9c9a-290fda5d5e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:19:30.229764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518a2bb-2db9-4477-9c9a-290fda5d5e04 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:19:30.247787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 585/1161] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:19:32.427938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d603a76-e391-4423-97c4-3db04d6679e8 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:32.428500 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d603a76-e391-4423-97c4-3db04d6679e8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:32.428500 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d603a76-e391-4423-97c4-3db04d6679e8 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:32.429860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 577/1162] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:19:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:19:32.435366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555affd6-4b67-4e03-9df9-79f9b42e6fb8 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:32.435764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555affd6-4b67-4e03-9df9-79f9b42e6fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:32.435925 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555affd6-4b67-4e03-9df9-79f9b42e6fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:32.502606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-555affd6-4b67-4e03-9df9-79f9b42e6fb8 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:19:32.503836 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-555affd6-4b67-4e03-9df9-79f9b42e6fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:32.600294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 586/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:19:32.898752 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d603a76-e391-4423-97c4-3db04d6679e8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:19:32.900343 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4d603a76-e391-4423-97c4-3db04d6679e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:33.002680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d603a76-e391-4423-97c4-3db04d6679e8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:19:33.073017 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:33.074463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd9327-e879-4d15-ba84-08775a0cd7a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:19:33.075150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd9327-e879-4d15-ba84-08775a0cd7a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:33.075504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd9327-e879-4d15-ba84-08775a0cd7a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:33.075708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd9327-e879-4d15-ba84-08775a0cd7a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:33.077821 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-20fd9327-e879-4d15-ba84-08775a0cd7a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:33.187454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd9327-e879-4d15-ba84-08775a0cd7a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:19:33.188886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd9327-e879-4d15-ba84-08775a0cd7a7 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:19:33.207013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 578/1164] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:19:35.460393 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3febf80-b5c0-42c7-a714-db0ae20ab311 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:35.460933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3febf80-b5c0-42c7-a714-db0ae20ab311 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:35.460933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3febf80-b5c0-42c7-a714-db0ae20ab311 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:35.462302 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 587/1165] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:19:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:19:35.467880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a68351-6862-48f5-9334-bd3a37d0f60a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:35.468317 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a68351-6862-48f5-9334-bd3a37d0f60a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:35.468427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a68351-6862-48f5-9334-bd3a37d0f60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:35.532748 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-22a68351-6862-48f5-9334-bd3a37d0f60a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:19:35.533940 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-22a68351-6862-48f5-9334-bd3a37d0f60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:35.585645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:35.630924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 579/1166] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:19:35.922182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3febf80-b5c0-42c7-a714-db0ae20ab311 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:19:35.923732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f3febf80-b5c0-42c7-a714-db0ae20ab311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:35.960122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:36.027616 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3febf80-b5c0-42c7-a714-db0ae20ab311 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:19:36.099014 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1c516-1840-4fb5-b598-8359ed183a59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:19:36.099676 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1c516-1840-4fb5-b598-8359ed183a59 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:36.099948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1c516-1840-4fb5-b598-8359ed183a59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:36.100219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1c516-1840-4fb5-b598-8359ed183a59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:36.101789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-15d1c516-1840-4fb5-b598-8359ed183a59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:36.181260 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:36.210860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1c516-1840-4fb5-b598-8359ed183a59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:19:36.212342 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1c516-1840-4fb5-b598-8359ed183a59 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:19:36.229993 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 588/1167] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:19:38.621815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7f6fb-cdf3-4fbc-8589-52739cf6bf8d None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:38.622354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7f6fb-cdf3-4fbc-8589-52739cf6bf8d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:38.622354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7f6fb-cdf3-4fbc-8589-52739cf6bf8d None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:38.623770 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 580/1168] 10.222.0.18 () {54 vars in 927 bytes} [Fri Oct 14 02:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:19:38.629154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53593893-51fb-4e86-92ad-02354bb1ecbc None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:38.629575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53593893-51fb-4e86-92ad-02354bb1ecbc None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:38.629702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53593893-51fb-4e86-92ad-02354bb1ecbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:38.694297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-53593893-51fb-4e86-92ad-02354bb1ecbc None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:19:38.695501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-53593893-51fb-4e86-92ad-02354bb1ecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:38.791360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 589/1169] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Oct 14 02:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:19:39.098322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7f6fb-cdf3-4fbc-8589-52739cf6bf8d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:19:39.100598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-fbb7f6fb-cdf3-4fbc-8589-52739cf6bf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:39.201876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7f6fb-cdf3-4fbc-8589-52739cf6bf8d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:19:39.273802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7f989-106b-41b9-9dfe-e84eb17a6779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:19:39.274402 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7f989-106b-41b9-9dfe-e84eb17a6779 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:19:39.274677 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7f989-106b-41b9-9dfe-e84eb17a6779 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:19:39.274946 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7f989-106b-41b9-9dfe-e84eb17a6779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:19:39.276665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-82a7f989-106b-41b9-9dfe-e84eb17a6779 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:19:39.284893 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:39.385789 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7f989-106b-41b9-9dfe-e84eb17a6779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:19:39.387190 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7f989-106b-41b9-9dfe-e84eb17a6779 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:19:39.404718 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 581/1170] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:19:48.789825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=82 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:19:49.402284 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=50 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:20:56.102860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e7c3d-09ff-4323-8043-7f9ce4a2b107 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:56.103513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e7c3d-09ff-4323-8043-7f9ce4a2b107 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:56.103513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e7c3d-09ff-4323-8043-7f9ce4a2b107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:56.179991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5a6e7c3d-09ff-4323-8043-7f9ce4a2b107 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:20:56.181212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5a6e7c3d-09ff-4323-8043-7f9ce4a2b107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:56.273552 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 590/1171] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:20:56.281454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116519a4-3955-4c8b-842e-da96cb25476c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:56.281933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116519a4-3955-4c8b-842e-da96cb25476c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:56.282046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116519a4-3955-4c8b-842e-da96cb25476c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:56.355983 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-116519a4-3955-4c8b-842e-da96cb25476c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:20:56.357100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-116519a4-3955-4c8b-842e-da96cb25476c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:56.453023 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 582/1172] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:20:56.457523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e7c3d-09ff-4323-8043-7f9ce4a2b107 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:20:56.459831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5a6e7c3d-09ff-4323-8043-7f9ce4a2b107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:56.471254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e7c3d-09ff-4323-8043-7f9ce4a2b107 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:20:56.533414 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:20:56.568486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32aafe-d346-4676-9288-e96d82a20a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:20:56.569093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32aafe-d346-4676-9288-e96d82a20a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:56.569397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32aafe-d346-4676-9288-e96d82a20a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:56.569696 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32aafe-d346-4676-9288-e96d82a20a8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:56.570386 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32aafe-d346-4676-9288-e96d82a20a8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:20:56.570715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32aafe-d346-4676-9288-e96d82a20a8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:20:56.571867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-8c32aafe-d346-4676-9288-e96d82a20a8a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:20:56.571867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 14 02:20:56.571867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: Oct 14 02:20:56.572866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32aafe-d346-4676-9288-e96d82a20a8a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:20:56.585323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 591/1173] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:20:56.594914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116519a4-3955-4c8b-842e-da96cb25476c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:20:56.604652 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116519a4-3955-4c8b-842e-da96cb25476c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:20:56.666976 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:20:56.699689 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5620f4-92ce-487b-936d-afafd06e771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:20:56.700302 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5620f4-92ce-487b-936d-afafd06e771c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:56.700581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5620f4-92ce-487b-936d-afafd06e771c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:56.700852 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5620f4-92ce-487b-936d-afafd06e771c admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:56.701476 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5620f4-92ce-487b-936d-afafd06e771c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:20:56.702516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-5d5620f4-92ce-487b-936d-afafd06e771c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:20:56.702516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 14 02:20:56.702516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: Oct 14 02:20:56.703765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5620f4-92ce-487b-936d-afafd06e771c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:20:56.759495 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 583/1174] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:20:56 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:20:56.763600 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32aafe-d346-4676-9288-e96d82a20a8a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:20:56.765251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8c32aafe-d346-4676-9288-e96d82a20a8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:56.799605 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32aafe-d346-4676-9288-e96d82a20a8a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:20:56.891117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211f234-04ac-4cb9-8633-971a136d94b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:20:56.891728 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f234-04ac-4cb9-8633-971a136d94b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:56.892001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f234-04ac-4cb9-8633-971a136d94b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:56.892271 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f234-04ac-4cb9-8633-971a136d94b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:56.892902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211f234-04ac-4cb9-8633-971a136d94b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:20:56.893908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-5211f234-04ac-4cb9-8633-971a136d94b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:20:56.893908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 14 02:20:56.893908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: Oct 14 02:20:56.895133 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211f234-04ac-4cb9-8633-971a136d94b9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:20:56.936917 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 592/1175] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:20:56 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:20:56.940902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5620f4-92ce-487b-936d-afafd06e771c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:20:56.942940 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5d5620f4-92ce-487b-936d-afafd06e771c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:57.003746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5620f4-92ce-487b-936d-afafd06e771c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:20:57.087313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:20:57.098142 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2eb512-c5e7-42c2-b0b9-fb18c781f4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:20:57.098764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2eb512-c5e7-42c2-b0b9-fb18c781f4a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:57.099041 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2eb512-c5e7-42c2-b0b9-fb18c781f4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:57.099343 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2eb512-c5e7-42c2-b0b9-fb18c781f4a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:57.100010 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2eb512-c5e7-42c2-b0b9-fb18c781f4a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:20:57.101622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2eb512-c5e7-42c2-b0b9-fb18c781f4a0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:20:57.113837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 584/1176] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:20:57.118249 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211f234-04ac-4cb9-8633-971a136d94b9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:20:57.119799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5211f234-04ac-4cb9-8633-971a136d94b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:57.131414 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211f234-04ac-4cb9-8633-971a136d94b9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:20:57.156012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:20:57.226706 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455e42c-e9b1-467a-82a7-34c2880e18b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:20:57.227656 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455e42c-e9b1-467a-82a7-34c2880e18b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:57.227937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455e42c-e9b1-467a-82a7-34c2880e18b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:57.228341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455e42c-e9b1-467a-82a7-34c2880e18b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:57.230170 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455e42c-e9b1-467a-82a7-34c2880e18b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:20:57.231963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455e42c-e9b1-467a-82a7-34c2880e18b9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:20:57.245077 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 593/1177] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:20:57.249187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2eb512-c5e7-42c2-b0b9-fb18c781f4a0 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:20:57.250731 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0b2eb512-c5e7-42c2-b0b9-fb18c781f4a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:57.284376 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2eb512-c5e7-42c2-b0b9-fb18c781f4a0 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:20:57.377859 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc5efc-0bd8-49b3-98d0-9b0bc3369f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:20:57.378519 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc5efc-0bd8-49b3-98d0-9b0bc3369f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:57.378798 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc5efc-0bd8-49b3-98d0-9b0bc3369f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:57.379091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc5efc-0bd8-49b3-98d0-9b0bc3369f96 admin admin] PATH_INFO: `/v3/projects/240fca63042d4b6781bae3ed85dc7f24/users/f5c432e3d3364ddbb3c914ba2b5f8cb7/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:57.401979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc5efc-0bd8-49b3-98d0-9b0bc3369f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=240fca63042d4b6781bae3ed85dc7f24, user_id=f5c432e3d3364ddbb3c914ba2b5f8cb7, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:20:57.404703 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc5efc-0bd8-49b3-98d0-9b0bc3369f96 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:20:57.429623 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 585/1178] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:20:57 2022] PUT /identity/v3/projects/240fca63042d4b6781bae3ed85dc7f24/users/f5c432e3d3364ddbb3c914ba2b5f8cb7/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:20:57.440712 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6ff89-0219-48e2-8e3c-6e998deae832 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:57.441004 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6ff89-0219-48e2-8e3c-6e998deae832 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:57.441288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6ff89-0219-48e2-8e3c-6e998deae832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:57.499744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3bb6ff89-0219-48e2-8e3c-6e998deae832 None None] MFA Rules not processed for user `f5c432e3d3364ddbb3c914ba2b5f8cb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:20:57.500933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3bb6ff89-0219-48e2-8e3c-6e998deae832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:57.575997 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 594/1179] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:20:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:20:57.589361 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7973a59-01dd-4837-853d-2578ea604f75 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:57.589645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7973a59-01dd-4837-853d-2578ea604f75 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:57.589919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7973a59-01dd-4837-853d-2578ea604f75 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:57.591275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 586/1180] 10.222.0.18 () {54 vars in 963 bytes} [Fri Oct 14 02:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:20:57.595866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd6fb1-fbf2-40e1-b1fe-7567930c352e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:57.596150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd6fb1-fbf2-40e1-b1fe-7567930c352e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:57.596421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd6fb1-fbf2-40e1-b1fe-7567930c352e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:57.628751 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:20:57.672323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-74dd6fb1-fbf2-40e1-b1fe-7567930c352e None None] MFA Rules not processed for user `eb7a43e284e1401688b2e202c67a5eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:20:57.673530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-74dd6fb1-fbf2-40e1-b1fe-7567930c352e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:57.749362 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 595/1181] 10.222.0.18 () {58 vars in 1060 bytes} [Fri Oct 14 02:20:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:20:57.757979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7973a59-01dd-4837-853d-2578ea604f75 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:20:57.759469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c7973a59-01dd-4837-853d-2578ea604f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:57.809179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:20:57.862764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7973a59-01dd-4837-853d-2578ea604f75 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:20:57.936663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13c82b-44f1-4617-8067-75fa8163ccec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:20:57.937306 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13c82b-44f1-4617-8067-75fa8163ccec service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:57.937587 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13c82b-44f1-4617-8067-75fa8163ccec service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:57.937863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13c82b-44f1-4617-8067-75fa8163ccec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:57.939496 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5d13c82b-44f1-4617-8067-75fa8163ccec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:58.039902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13c82b-44f1-4617-8067-75fa8163ccec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:20:58.041257 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13c82b-44f1-4617-8067-75fa8163ccec service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:20:58.044730 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:20:58.060039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 587/1182] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:20:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:20:58.105290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353dbd3-7dc1-4ce4-9682-83bd88d9324f None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:58.105582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353dbd3-7dc1-4ce4-9682-83bd88d9324f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:58.105862 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353dbd3-7dc1-4ce4-9682-83bd88d9324f None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:58.107158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 596/1183] 10.222.0.18 () {52 vars in 830 bytes} [Fri Oct 14 02:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:20:58.110730 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13c82b-44f1-4617-8067-75fa8163ccec service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:20:58.111977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5d13c82b-44f1-4617-8067-75fa8163ccec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:58.209035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13c82b-44f1-4617-8067-75fa8163ccec service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:20:58.299530 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:20:58.300130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:58.300377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:58.300648 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] PATH_INFO: `/v3/users/f5c432e3d3364ddbb3c914ba2b5f8cb7` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:58.314091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c432e3d3364ddbb3c914ba2b5f8cb7)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:20:58.315134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:20:58.315134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 14 02:20:58.315134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: Oct 14 02:20:58.316379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:20:58.326409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:20:58.382881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c432e3d3364ddbb3c914ba2b5f8cb7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:20:58.391946 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c432e3d3364ddbb3c914ba2b5f8cb7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:20:58.392430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c432e3d3364ddbb3c914ba2b5f8cb7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:20:58.417426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c432e3d3364ddbb3c914ba2b5f8cb7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:20:58.431521 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c432e3d3364ddbb3c914ba2b5f8cb7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:20:58.437384 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 588/1184] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:20:58 2022] DELETE /identity/v3/users/f5c432e3d3364ddbb3c914ba2b5f8cb7 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:20:58.441904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8afc07-0450-499d-9804-918600a85c88 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:58.442435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8afc07-0450-499d-9804-918600a85c88 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:58.442546 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8afc07-0450-499d-9804-918600a85c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:58.502042 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-aa8afc07-0450-499d-9804-918600a85c88 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:20:58.503269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-aa8afc07-0450-499d-9804-918600a85c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:58.578427 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:20:58.591309 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 597/1185] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:20:58.602113 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:20:58.603356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:58.700725 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309faa7-015c-4e6f-8649-d784dc38a6ec admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:20:58.758112 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:20:58.777820 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db5e48-64d0-4700-8dcf-5418773105b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:20:58.778440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db5e48-64d0-4700-8dcf-5418773105b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:58.778717 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db5e48-64d0-4700-8dcf-5418773105b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:58.778988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db5e48-64d0-4700-8dcf-5418773105b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:58.780600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-83db5e48-64d0-4700-8dcf-5418773105b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:58.888261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db5e48-64d0-4700-8dcf-5418773105b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:20:58.889623 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db5e48-64d0-4700-8dcf-5418773105b7 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:20:58.907673 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 589/1186] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:20:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:20:59.340121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8afc07-0450-499d-9804-918600a85c88 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:20:59.341358 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-aa8afc07-0450-499d-9804-918600a85c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:20:59.353195 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8afc07-0450-499d-9804-918600a85c88 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:20:59.449591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:20:59.450322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:20:59.450482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:20:59.450748 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d admin admin] PATH_INFO: `/v3/projects/240fca63042d4b6781bae3ed85dc7f24` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:20:59.460947 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d admin admin] RBAC: Authorizing `identity:delete_project(project_id=240fca63042d4b6781bae3ed85dc7f24)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:20:59.462008 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:20:59.462008 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 14 02:20:59.462008 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: Oct 14 02:20:59.463251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:20:59.520090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240fca63042d4b6781bae3ed85dc7f24', 'request_id': 'req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:20:59.526246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240fca63042d4b6781bae3ed85dc7f24', 'request_id': 'req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:20:59.539750 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240fca63042d4b6781bae3ed85dc7f24', 'request_id': 'req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:20:59.566062 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d admin admin] The token cache is being invalidate because project 240fca63042d4b6781bae3ed85dc7f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:20:59.566348 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:20:59.566858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ace073e1-01c7-4325-b8f5-cd1d1e2be96d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:20:59.569115 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 598/1187] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:20:59 2022] DELETE /identity/v3/projects/240fca63042d4b6781bae3ed85dc7f24 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:21:08.905424 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=62 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:21:09.564115 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:22:24.174430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779206ed-36d7-401a-8014-aeefaa471ae8 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:24.175014 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779206ed-36d7-401a-8014-aeefaa471ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:24.175014 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779206ed-36d7-401a-8014-aeefaa471ae8 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:24.176376 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 590/1188] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 02:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:22:24.181720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61bb0c-4cf2-4504-934d-82126ecee5c6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:24.182173 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61bb0c-4cf2-4504-934d-82126ecee5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:24.182284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61bb0c-4cf2-4504-934d-82126ecee5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:24.262345 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ad61bb0c-4cf2-4504-934d-82126ecee5c6 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:22:24.263568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ad61bb0c-4cf2-4504-934d-82126ecee5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:22:24.363561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 599/1189] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 02:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:22:24.522331 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779206ed-36d7-401a-8014-aeefaa471ae8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:22:24.524469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-779206ed-36d7-401a-8014-aeefaa471ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:22:24.643036 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779206ed-36d7-401a-8014-aeefaa471ae8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:22:24.648511 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6e898-0ebc-43e4-afd3-8549d82b6cda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:22:24.649129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6e898-0ebc-43e4-afd3-8549d82b6cda service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:24.649411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6e898-0ebc-43e4-afd3-8549d82b6cda service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:24.649686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6e898-0ebc-43e4-afd3-8549d82b6cda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:24.657605 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6e898-0ebc-43e4-afd3-8549d82b6cda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:22:24.659268 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6e898-0ebc-43e4-afd3-8549d82b6cda service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:22:24.669184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 591/1190] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:22:25.775685 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07810f-03d1-4c2f-bb30-414b7b43ab2a None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:25.776165 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07810f-03d1-4c2f-bb30-414b7b43ab2a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:25.776287 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07810f-03d1-4c2f-bb30-414b7b43ab2a None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:25.777736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 600/1191] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 02:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:22:25.784244 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3783d-fb02-446b-b651-57f51b9ca2b4 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:25.784707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3783d-fb02-446b-b651-57f51b9ca2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:25.784801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3783d-fb02-446b-b651-57f51b9ca2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:25.805523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f0d3783d-fb02-446b-b651-57f51b9ca2b4 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:22:25.806729 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f0d3783d-fb02-446b-b651-57f51b9ca2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:22:25.814652 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 592/1192] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 02:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:22:25.969777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07810f-03d1-4c2f-bb30-414b7b43ab2a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:22:25.984516 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07810f-03d1-4c2f-bb30-414b7b43ab2a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:22:25.989806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32867d32-7ff8-4f16-90f8-b92a4841274c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:22:25.990424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32867d32-7ff8-4f16-90f8-b92a4841274c service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:25.990945 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32867d32-7ff8-4f16-90f8-b92a4841274c service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:25.991250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32867d32-7ff8-4f16-90f8-b92a4841274c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:25.999823 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32867d32-7ff8-4f16-90f8-b92a4841274c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:22:26.001471 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32867d32-7ff8-4f16-90f8-b92a4841274c service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:22:26.011052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 601/1193] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:22:26.293125 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a425a3-4ee6-4371-810c-6d55dfabed26 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:26.293342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a425a3-4ee6-4371-810c-6d55dfabed26 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:26.293642 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a425a3-4ee6-4371-810c-6d55dfabed26 None None] PATH_INFO: `/v3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:26.294933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 593/1194] 10.222.0.25 () {54 vars in 925 bytes} [Fri Oct 14 02:22:26 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:22:26.310110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e00de-13b8-4641-a176-c08d22ac4e67 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:26.310394 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e00de-13b8-4641-a176-c08d22ac4e67 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:26.310668 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e00de-13b8-4641-a176-c08d22ac4e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:26.370736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d37e00de-13b8-4641-a176-c08d22ac4e67 None None] MFA Rules not processed for user `5e935ba63cdd4256874b99a2eb0a1a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:22:26.371953 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d37e00de-13b8-4641-a176-c08d22ac4e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:22:26.388129 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:22:26.440847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 602/1195] 10.222.0.25 () {58 vars in 1015 bytes} [Fri Oct 14 02:22:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:22:29.991916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b9c07-4963-4aa4-8ada-298451bbfaa4 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:29.992971 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b9c07-4963-4aa4-8ada-298451bbfaa4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:29.993166 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b9c07-4963-4aa4-8ada-298451bbfaa4 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:29.994477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 594/1196] 10.222.0.18 () {54 vars in 961 bytes} [Fri Oct 14 02:22:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:22:29.999374 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a317188-8fa4-458b-9770-64be5f795101 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:29.999786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a317188-8fa4-458b-9770-64be5f795101 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:29.999931 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a317188-8fa4-458b-9770-64be5f795101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:30.019755 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0a317188-8fa4-458b-9770-64be5f795101 None None] MFA Rules not processed for user `5e935ba63cdd4256874b99a2eb0a1a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:22:30.020971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0a317188-8fa4-458b-9770-64be5f795101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:22:30.030261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 603/1197] 10.222.0.18 () {58 vars in 1058 bytes} [Fri Oct 14 02:22:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:22:30.038087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b9c07-4963-4aa4-8ada-298451bbfaa4 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:22:30.052108 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b9c07-4963-4aa4-8ada-298451bbfaa4 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:22:30.057774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb7f11-99cd-4754-bbc3-6a228e6fbef9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e935ba63cdd4256874b99a2eb0a1a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:22:30.058375 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb7f11-99cd-4754-bbc3-6a228e6fbef9 service placement] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:30.058657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb7f11-99cd-4754-bbc3-6a228e6fbef9 service placement] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:30.058943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb7f11-99cd-4754-bbc3-6a228e6fbef9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:30.067038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb7f11-99cd-4754-bbc3-6a228e6fbef9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:22:30.068670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb7f11-99cd-4754-bbc3-6a228e6fbef9 service placement] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:22:30.078549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 595/1198] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:22:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:22:37.452966 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb812d2-6d2a-4d00-872c-6b6f5a302dc4 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:37.453568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb812d2-6d2a-4d00-872c-6b6f5a302dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:37.453568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb812d2-6d2a-4d00-872c-6b6f5a302dc4 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:37.454865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 604/1199] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 02:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:22:37.461461 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaf13e-87b0-4970-819e-c6b52deee925 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:37.461909 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaf13e-87b0-4970-819e-c6b52deee925 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:37.462026 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaf13e-87b0-4970-819e-c6b52deee925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:37.481904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e7aaf13e-87b0-4970-819e-c6b52deee925 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:22:37.483130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e7aaf13e-87b0-4970-819e-c6b52deee925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:22:37.491115 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 596/1200] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 02:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:22:37.650826 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb812d2-6d2a-4d00-872c-6b6f5a302dc4 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:22:37.666704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb812d2-6d2a-4d00-872c-6b6f5a302dc4 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:22:37.675613 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077316a-59b1-4415-8d01-f9da11320822 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:22:37.676243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077316a-59b1-4415-8d01-f9da11320822 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:37.676522 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077316a-59b1-4415-8d01-f9da11320822 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:37.676794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077316a-59b1-4415-8d01-f9da11320822 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:37.685486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077316a-59b1-4415-8d01-f9da11320822 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:22:37.687057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077316a-59b1-4415-8d01-f9da11320822 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:22:37.697512 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 605/1201] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:22:38.771188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a56c5d-e355-4d5b-bbc7-b8f6048a226a None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:38.771728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a56c5d-e355-4d5b-bbc7-b8f6048a226a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:38.771728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a56c5d-e355-4d5b-bbc7-b8f6048a226a None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:38.773065 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 597/1202] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 02:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:22:38.779304 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d81973-a74a-407c-9dac-1531ac17ce07 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:38.779724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d81973-a74a-407c-9dac-1531ac17ce07 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:38.779836 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d81973-a74a-407c-9dac-1531ac17ce07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:38.799401 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b0d81973-a74a-407c-9dac-1531ac17ce07 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:22:38.800573 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b0d81973-a74a-407c-9dac-1531ac17ce07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:22:38.808802 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 606/1203] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 02:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:22:38.957778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a56c5d-e355-4d5b-bbc7-b8f6048a226a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:22:38.972183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a56c5d-e355-4d5b-bbc7-b8f6048a226a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:22:38.977993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b26da-b073-42b3-92e9-bff55e3a3be8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:22:38.978669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b26da-b073-42b3-92e9-bff55e3a3be8 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:38.978875 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b26da-b073-42b3-92e9-bff55e3a3be8 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:38.979182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b26da-b073-42b3-92e9-bff55e3a3be8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:38.987014 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b26da-b073-42b3-92e9-bff55e3a3be8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:22:38.988621 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b26da-b073-42b3-92e9-bff55e3a3be8 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:22:38.998856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 598/1204] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:22:42.997160 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cad0d1-d4f8-461f-98d3-2d631900b3fa None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:42.997677 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cad0d1-d4f8-461f-98d3-2d631900b3fa None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:42.997799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cad0d1-d4f8-461f-98d3-2d631900b3fa None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:42.999047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 607/1205] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:22:43.004197 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722df1e-7c2d-47f1-9c9c-b9f17b95176d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:43.004643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722df1e-7c2d-47f1-9c9c-b9f17b95176d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:43.004749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722df1e-7c2d-47f1-9c9c-b9f17b95176d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:43.024881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-3722df1e-7c2d-47f1-9c9c-b9f17b95176d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:22:43.026082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3722df1e-7c2d-47f1-9c9c-b9f17b95176d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:22:43.033766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 599/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:22:43.321327 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cad0d1-d4f8-461f-98d3-2d631900b3fa None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:22:43.323025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f9cad0d1-d4f8-461f-98d3-2d631900b3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:22:43.400112 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cad0d1-d4f8-461f-98d3-2d631900b3fa None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:22:43.403649 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9047529-b197-4920-ad1a-ccbe16256799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:22:43.404252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9047529-b197-4920-ad1a-ccbe16256799 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:43.404527 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9047529-b197-4920-ad1a-ccbe16256799 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:43.404795 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9047529-b197-4920-ad1a-ccbe16256799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:43.413690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9047529-b197-4920-ad1a-ccbe16256799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:22:43.415132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9047529-b197-4920-ad1a-ccbe16256799 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:22:43.424315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 608/1207] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:22:45.558414 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a66759-f5ef-45c6-bb37-ff21551e7a48 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:45.558931 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a66759-f5ef-45c6-bb37-ff21551e7a48 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:45.559035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a66759-f5ef-45c6-bb37-ff21551e7a48 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:45.560333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 600/1208] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:22:45.565730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086325eb-0729-4b51-9cb7-c6ab0a6a12b0 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:45.566114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086325eb-0729-4b51-9cb7-c6ab0a6a12b0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:45.566281 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086325eb-0729-4b51-9cb7-c6ab0a6a12b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:45.586904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-086325eb-0729-4b51-9cb7-c6ab0a6a12b0 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:22:45.588164 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-086325eb-0729-4b51-9cb7-c6ab0a6a12b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:22:45.596752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 609/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:22:45.887406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a66759-f5ef-45c6-bb37-ff21551e7a48 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:22:45.897949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a66759-f5ef-45c6-bb37-ff21551e7a48 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:22:45.901486 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348a341-2005-4672-9e1c-59afb7898d09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:22:45.902089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a341-2005-4672-9e1c-59afb7898d09 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:45.902370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a341-2005-4672-9e1c-59afb7898d09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:45.902636 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a341-2005-4672-9e1c-59afb7898d09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:45.911440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348a341-2005-4672-9e1c-59afb7898d09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:22:45.912847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348a341-2005-4672-9e1c-59afb7898d09 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:22:45.922270 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 601/1210] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:22:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:22:49.623322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdeaf6-5959-410c-9355-80b1fb33295b None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:49.623856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdeaf6-5959-410c-9355-80b1fb33295b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:49.623856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdeaf6-5959-410c-9355-80b1fb33295b None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:49.625197 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 610/1211] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:22:49.630353 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb65146-6eea-49fe-b4f1-2cc2a02c847d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:49.630769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb65146-6eea-49fe-b4f1-2cc2a02c847d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:49.630973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb65146-6eea-49fe-b4f1-2cc2a02c847d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:49.638936 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=81 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:22:49.652153 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4bb65146-6eea-49fe-b4f1-2cc2a02c847d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:22:49.653338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4bb65146-6eea-49fe-b4f1-2cc2a02c847d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:22:49.661080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 602/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:22:49.970130 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccdeaf6-5959-410c-9355-80b1fb33295b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:22:49.971859 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1ccdeaf6-5959-410c-9355-80b1fb33295b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:22:50.036373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:22:50.049630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccdeaf6-5959-410c-9355-80b1fb33295b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:22:50.053199 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30804c-6afb-4960-a29a-58e528cc862b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:22:50.053822 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30804c-6afb-4960-a29a-58e528cc862b service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:50.054226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30804c-6afb-4960-a29a-58e528cc862b service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:50.054507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30804c-6afb-4960-a29a-58e528cc862b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:50.063366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30804c-6afb-4960-a29a-58e528cc862b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:22:50.064808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30804c-6afb-4960-a29a-58e528cc862b service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:22:50.074602 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 611/1213] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:22:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:22:52.193016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85209f6d-abad-4eda-b69f-40d0f54d5dff None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:52.193581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85209f6d-abad-4eda-b69f-40d0f54d5dff None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:52.193581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85209f6d-abad-4eda-b69f-40d0f54d5dff None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:52.195771 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 603/1214] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 02:22:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:22:52.201007 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06514c2d-41a3-420d-914a-978e7d173e90 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:52.201453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06514c2d-41a3-420d-914a-978e7d173e90 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:52.201581 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06514c2d-41a3-420d-914a-978e7d173e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:52.221153 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-06514c2d-41a3-420d-914a-978e7d173e90 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:22:52.222342 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-06514c2d-41a3-420d-914a-978e7d173e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:22:52.230211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 612/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 02:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:22:52.532391 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85209f6d-abad-4eda-b69f-40d0f54d5dff None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:22:52.534106 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-85209f6d-abad-4eda-b69f-40d0f54d5dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:22:52.547499 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85209f6d-abad-4eda-b69f-40d0f54d5dff None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:22:52.550968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce97c9b-88e4-4c1a-98a3-4802fddf4c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:22:52.551595 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce97c9b-88e4-4c1a-98a3-4802fddf4c33 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:22:52.551877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce97c9b-88e4-4c1a-98a3-4802fddf4c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:22:52.552152 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce97c9b-88e4-4c1a-98a3-4802fddf4c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:22:52.561798 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce97c9b-88e4-4c1a-98a3-4802fddf4c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:22:52.563259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce97c9b-88e4-4c1a-98a3-4802fddf4c33 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:22:52.572792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 604/1216] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:23:02.216453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=11 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:23:02.568354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=8 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:16.549194 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e89899-eb38-4176-93bd-d9467fef05e1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:16.550316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e89899-eb38-4176-93bd-d9467fef05e1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:16.550832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e89899-eb38-4176-93bd-d9467fef05e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:16.580669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-46e89899-eb38-4176-93bd-d9467fef05e1 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:16.582113 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-46e89899-eb38-4176-93bd-d9467fef05e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:16.595270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 613/1217] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:16.605882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1c28e-a96d-45b9-b2d4-2601a4029e69 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:16.606510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1c28e-a96d-45b9-b2d4-2601a4029e69 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:16.606510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1c28e-a96d-45b9-b2d4-2601a4029e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:16.644310 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a7f1c28e-a96d-45b9-b2d4-2601a4029e69 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:16.645593 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a7f1c28e-a96d-45b9-b2d4-2601a4029e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:16.659437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 605/1218] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:16.664917 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e89899-eb38-4176-93bd-d9467fef05e1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:16.668930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7d442-291a-47c0-a381-1d8f9bf8c649 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:16.669227 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7d442-291a-47c0-a381-1d8f9bf8c649 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:16.669487 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7d442-291a-47c0-a381-1d8f9bf8c649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:16.678798 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e89899-eb38-4176-93bd-d9467fef05e1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:16.688260 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d926c-2ba1-49fa-b257-0b907b0f653d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:16.688898 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d926c-2ba1-49fa-b257-0b907b0f653d admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:16.689175 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d926c-2ba1-49fa-b257-0b907b0f653d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:16.689447 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d926c-2ba1-49fa-b257-0b907b0f653d admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:16.690353 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d926c-2ba1-49fa-b257-0b907b0f653d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:16.690677 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d926c-2ba1-49fa-b257-0b907b0f653d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:16.694582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d926c-2ba1-49fa-b257-0b907b0f653d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:16.704775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-eed7d442-291a-47c0-a381-1d8f9bf8c649 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:16.706015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-eed7d442-291a-47c0-a381-1d8f9bf8c649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:16.716807 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 614/1219] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:16.721490 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 606/1220] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:24:16.726167 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d926c-2ba1-49fa-b257-0b907b0f653d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:16.739426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b25254-9be1-480d-8281-aefa6714b6af None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:16.739794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b25254-9be1-480d-8281-aefa6714b6af None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:16.740072 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b25254-9be1-480d-8281-aefa6714b6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:16.741073 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d926c-2ba1-49fa-b257-0b907b0f653d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:16.751994 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3367b-d67d-4a39-8d0c-96fc6b2a079a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:16.752621 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3367b-d67d-4a39-8d0c-96fc6b2a079a admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:16.752897 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3367b-d67d-4a39-8d0c-96fc6b2a079a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:16.753166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3367b-d67d-4a39-8d0c-96fc6b2a079a admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:16.753814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3367b-d67d-4a39-8d0c-96fc6b2a079a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:16.755001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-9af3367b-d67d-4a39-8d0c-96fc6b2a079a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:24:16.755001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 14 02:24:16.755001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: Oct 14 02:24:16.756394 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3367b-d67d-4a39-8d0c-96fc6b2a079a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:16.774053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-40b25254-9be1-480d-8281-aefa6714b6af None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:16.775340 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-40b25254-9be1-480d-8281-aefa6714b6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:16.793692 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 607/1221] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:24:16.799093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b25254-9be1-480d-8281-aefa6714b6af None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:16.815234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b25254-9be1-480d-8281-aefa6714b6af None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:16.835877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de954586-98b1-461e-b419-91d17352e431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:16.836534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de954586-98b1-461e-b419-91d17352e431 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:16.836858 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de954586-98b1-461e-b419-91d17352e431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:16.837170 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de954586-98b1-461e-b419-91d17352e431 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:16.837853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de954586-98b1-461e-b419-91d17352e431 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:16.838163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de954586-98b1-461e-b419-91d17352e431 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:16.839128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 615/1222] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:16 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:16.839255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-de954586-98b1-461e-b419-91d17352e431 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:24:16.839255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 14 02:24:16.839255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: Oct 14 02:24:16.841543 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de954586-98b1-461e-b419-91d17352e431 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:16.843657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3367b-d67d-4a39-8d0c-96fc6b2a079a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:16.859097 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 608/1223] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:16.872148 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de954586-98b1-461e-b419-91d17352e431 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:16.885781 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de954586-98b1-461e-b419-91d17352e431 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:16.967093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3367b-d67d-4a39-8d0c-96fc6b2a079a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:16.973330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd407369-7853-4968-a068-d8f340ee1472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:16.974488 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd407369-7853-4968-a068-d8f340ee1472 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:16.974772 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd407369-7853-4968-a068-d8f340ee1472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:16.975042 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd407369-7853-4968-a068-d8f340ee1472 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:16.975829 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd407369-7853-4968-a068-d8f340ee1472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:16.977950 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd407369-7853-4968-a068-d8f340ee1472 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:16.984577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a24f1-4185-4309-aaa8-17c42da894ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:16.985149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a24f1-4185-4309-aaa8-17c42da894ec admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:16.985428 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a24f1-4185-4309-aaa8-17c42da894ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:16.985702 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a24f1-4185-4309-aaa8-17c42da894ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:16.986340 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a24f1-4185-4309-aaa8-17c42da894ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:16.988279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a24f1-4185-4309-aaa8-17c42da894ec admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.016372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 616/1224] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:16 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:24:17.017713 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0aa3b-b5be-4767-99c6-051eb57ff7d4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.018055 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0aa3b-b5be-4767-99c6-051eb57ff7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.018334 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0aa3b-b5be-4767-99c6-051eb57ff7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.051029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 609/1225] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:16 2022] POST /identity/v3/projects => generated 357 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:17.052501 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c096b6-701e-42c1-a1ed-fc1401b12559 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.052787 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c096b6-701e-42c1-a1ed-fc1401b12559 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.053064 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c096b6-701e-42c1-a1ed-fc1401b12559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.061053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-70a0aa3b-b5be-4767-99c6-051eb57ff7d4 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:17.063328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-70a0aa3b-b5be-4767-99c6-051eb57ff7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:17.074275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-72c096b6-701e-42c1-a1ed-fc1401b12559 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:17.075507 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-72c096b6-701e-42c1-a1ed-fc1401b12559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:17.132090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 617/1226] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:17.132090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f68968-0f2a-410a-ab1f-1303a30132e9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.132090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f68968-0f2a-410a-ab1f-1303a30132e9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.132090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f68968-0f2a-410a-ab1f-1303a30132e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.148308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 610/1227] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:17.152951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c096b6-701e-42c1-a1ed-fc1401b12559 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.153126 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b5f68968-0f2a-410a-ab1f-1303a30132e9 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:17.153126 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b5f68968-0f2a-410a-ab1f-1303a30132e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:17.158581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c096b6-701e-42c1-a1ed-fc1401b12559 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.160756 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 618/1228] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:17.162009 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f051b8-292a-49fd-b56a-1bb5bc43f959 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.162291 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f051b8-292a-49fd-b56a-1bb5bc43f959 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.162562 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f051b8-292a-49fd-b56a-1bb5bc43f959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.163465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2650f9-0a72-4206-a35b-2b064590935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.164073 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2650f9-0a72-4206-a35b-2b064590935b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.164971 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2650f9-0a72-4206-a35b-2b064590935b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.165246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2650f9-0a72-4206-a35b-2b064590935b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.166020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2650f9-0a72-4206-a35b-2b064590935b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.167607 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2650f9-0a72-4206-a35b-2b064590935b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.180971 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 611/1229] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:17.180971 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176c9d1-03c9-4f77-9dac-78d31d11ea21 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.180971 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176c9d1-03c9-4f77-9dac-78d31d11ea21 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.180971 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176c9d1-03c9-4f77-9dac-78d31d11ea21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.182615 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-95f051b8-292a-49fd-b56a-1bb5bc43f959 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:17.183871 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-95f051b8-292a-49fd-b56a-1bb5bc43f959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:17.191936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 619/1230] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:17.193266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f051b8-292a-49fd-b56a-1bb5bc43f959 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.202080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c176c9d1-03c9-4f77-9dac-78d31d11ea21 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:17.203295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c176c9d1-03c9-4f77-9dac-78d31d11ea21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:17.206169 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f051b8-292a-49fd-b56a-1bb5bc43f959 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.210742 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce30de5-740e-42dc-a46f-52bfd58df7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.211376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce30de5-740e-42dc-a46f-52bfd58df7f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.211783 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 612/1231] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:17.211909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce30de5-740e-42dc-a46f-52bfd58df7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.212006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce30de5-740e-42dc-a46f-52bfd58df7f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.212654 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce30de5-740e-42dc-a46f-52bfd58df7f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.212774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcac04-31d2-4874-b89b-6bb26d709e7d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.213048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcac04-31d2-4874-b89b-6bb26d709e7d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.213320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcac04-31d2-4874-b89b-6bb26d709e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.214613 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce30de5-740e-42dc-a46f-52bfd58df7f9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.233378 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-bbfcac04-31d2-4874-b89b-6bb26d709e7d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:17.234567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bbfcac04-31d2-4874-b89b-6bb26d709e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:17.240412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 620/1232] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/users => generated 279 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:17.240412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095cc2d-aa1f-4301-bda0-d1b890add93c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.240412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095cc2d-aa1f-4301-bda0-d1b890add93c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.240412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095cc2d-aa1f-4301-bda0-d1b890add93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.246845 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 613/1233] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:17.248267 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b8520-6eb2-45c8-900d-496ddd9cbdc2 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.248554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b8520-6eb2-45c8-900d-496ddd9cbdc2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.248829 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b8520-6eb2-45c8-900d-496ddd9cbdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.263320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0095cc2d-aa1f-4301-bda0-d1b890add93c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:17.264538 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0095cc2d-aa1f-4301-bda0-d1b890add93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:17.267655 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-761b8520-6eb2-45c8-900d-496ddd9cbdc2 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:17.268830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-761b8520-6eb2-45c8-900d-496ddd9cbdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:17.273075 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 621/1234] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:17.274261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fd361-c88c-4de5-9131-27db408524ed None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.274545 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fd361-c88c-4de5-9131-27db408524ed None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.274817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fd361-c88c-4de5-9131-27db408524ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.280296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 614/1235] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:17.281086 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b8520-6eb2-45c8-900d-496ddd9cbdc2 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.284200 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:17.292430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b8520-6eb2-45c8-900d-496ddd9cbdc2 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.295867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1f7fd361-c88c-4de5-9131-27db408524ed None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:17.297009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b761ec5-97ae-42ce-915b-765ddb6fee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.297149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1f7fd361-c88c-4de5-9131-27db408524ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:17.297618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b761ec5-97ae-42ce-915b-765ddb6fee93 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.297902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b761ec5-97ae-42ce-915b-765ddb6fee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.298177 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b761ec5-97ae-42ce-915b-765ddb6fee93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.298813 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b761ec5-97ae-42ce-915b-765ddb6fee93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.300441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b761ec5-97ae-42ce-915b-765ddb6fee93 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.305623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 622/1236] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:17.306809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46d59e-f343-48fd-8e4c-30ba7c0038b5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.307175 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46d59e-f343-48fd-8e4c-30ba7c0038b5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.307467 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46d59e-f343-48fd-8e4c-30ba7c0038b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.314558 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 615/1237] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:17.315878 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f426d-3bba-4868-b072-9c8eced38f59 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.316228 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f426d-3bba-4868-b072-9c8eced38f59 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.316566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f426d-3bba-4868-b072-9c8eced38f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.330433 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3b46d59e-f343-48fd-8e4c-30ba7c0038b5 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:17.331658 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3b46d59e-f343-48fd-8e4c-30ba7c0038b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:17.336076 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f07f426d-3bba-4868-b072-9c8eced38f59 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:17.337227 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f07f426d-3bba-4868-b072-9c8eced38f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:17.339703 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 623/1238] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:17.340483 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46d59e-f343-48fd-8e4c-30ba7c0038b5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.347189 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 616/1239] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:17.348577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f22f1-406c-44db-b0a7-1ea7eb6b8cc1 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.349579 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f22f1-406c-44db-b0a7-1ea7eb6b8cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.349890 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f22f1-406c-44db-b0a7-1ea7eb6b8cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.352809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46d59e-f343-48fd-8e4c-30ba7c0038b5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.357347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1686a-3494-411d-aa8b-f0ea457e279e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.358783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1686a-3494-411d-aa8b-f0ea457e279e admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.358948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:17.359045 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1686a-3494-411d-aa8b-f0ea457e279e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.359045 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1686a-3494-411d-aa8b-f0ea457e279e admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.359247 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1686a-3494-411d-aa8b-f0ea457e279e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.361107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1686a-3494-411d-aa8b-f0ea457e279e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.371469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7e8f22f1-406c-44db-b0a7-1ea7eb6b8cc1 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:17.372201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 624/1240] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:17.375217 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7e8f22f1-406c-44db-b0a7-1ea7eb6b8cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:17.375308 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1686a-3494-411d-aa8b-f0ea457e279e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.381709 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 617/1241] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:17.382724 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f22f1-406c-44db-b0a7-1ea7eb6b8cc1 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.383355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1686a-3494-411d-aa8b-f0ea457e279e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.389167 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7b34b-856b-4a10-a5db-bc1eb90807f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.389757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7b34b-856b-4a10-a5db-bc1eb90807f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.390033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7b34b-856b-4a10-a5db-bc1eb90807f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.390310 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7b34b-856b-4a10-a5db-bc1eb90807f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.390942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7b34b-856b-4a10-a5db-bc1eb90807f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:17.391290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7b34b-856b-4a10-a5db-bc1eb90807f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.392656 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f22f1-406c-44db-b0a7-1ea7eb6b8cc1 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.392892 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7b34b-856b-4a10-a5db-bc1eb90807f6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.397409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bd58f-a809-47f0-a81d-975e92cc3afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.398007 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bd58f-a809-47f0-a81d-975e92cc3afb admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.398286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bd58f-a809-47f0-a81d-975e92cc3afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.398560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bd58f-a809-47f0-a81d-975e92cc3afb admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.399240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bd58f-a809-47f0-a81d-975e92cc3afb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:17.399550 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bd58f-a809-47f0-a81d-975e92cc3afb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.401125 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bd58f-a809-47f0-a81d-975e92cc3afb admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.404991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 625/1242] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:17.405891 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7b34b-856b-4a10-a5db-bc1eb90807f6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.419624 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 618/1243] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:17.419624 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bd58f-a809-47f0-a81d-975e92cc3afb admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.422328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7b34b-856b-4a10-a5db-bc1eb90807f6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.423188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bd58f-a809-47f0-a81d-975e92cc3afb admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.426889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7037e15-6b3a-4302-8b77-0664628297fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.428084 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7037e15-6b3a-4302-8b77-0664628297fd admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.428365 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7037e15-6b3a-4302-8b77-0664628297fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.428476 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c746625-5df0-4c9e-825d-c62977190b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.428638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7037e15-6b3a-4302-8b77-0664628297fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.429080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c746625-5df0-4c9e-825d-c62977190b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.429280 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7037e15-6b3a-4302-8b77-0664628297fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:17.429382 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c746625-5df0-4c9e-825d-c62977190b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.429583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7037e15-6b3a-4302-8b77-0664628297fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.429686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c746625-5df0-4c9e-825d-c62977190b2d admin admin] PATH_INFO: `/v3/projects/31425e92610a4541b2e74b733f6b32e8/users/503eb61fecdf4509bb21dd453027ee79/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.431204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7037e15-6b3a-4302-8b77-0664628297fd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.442861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 626/1244] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:17.442861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7037e15-6b3a-4302-8b77-0664628297fd admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.447566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c746625-5df0-4c9e-825d-c62977190b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=31425e92610a4541b2e74b733f6b32e8, user_id=503eb61fecdf4509bb21dd453027ee79, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.447566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c746625-5df0-4c9e-825d-c62977190b2d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.456778 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7037e15-6b3a-4302-8b77-0664628297fd admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.462496 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 619/1245] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:17 2022] PUT /identity/v3/projects/31425e92610a4541b2e74b733f6b32e8/users/503eb61fecdf4509bb21dd453027ee79/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:17.463706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c746625-5df0-4c9e-825d-c62977190b2d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.519714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbfe3a-50bf-471f-99a2-68a049141dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.520487 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbfe3a-50bf-471f-99a2-68a049141dbe admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.521351 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbfe3a-50bf-471f-99a2-68a049141dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.521920 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbfe3a-50bf-471f-99a2-68a049141dbe admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.522590 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbfe3a-50bf-471f-99a2-68a049141dbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:17.522893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbfe3a-50bf-471f-99a2-68a049141dbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.524532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbfe3a-50bf-471f-99a2-68a049141dbe admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.534294 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c746625-5df0-4c9e-825d-c62977190b2d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.535733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 627/1246] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:17.537280 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbfe3a-50bf-471f-99a2-68a049141dbe admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.546746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbfe3a-50bf-471f-99a2-68a049141dbe admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.547180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad977f1-d4e5-4399-a144-e6a051b5820c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.547818 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad977f1-d4e5-4399-a144-e6a051b5820c admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.548101 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad977f1-d4e5-4399-a144-e6a051b5820c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.548404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad977f1-d4e5-4399-a144-e6a051b5820c admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.549185 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad977f1-d4e5-4399-a144-e6a051b5820c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:17.549500 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad977f1-d4e5-4399-a144-e6a051b5820c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.551358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad977f1-d4e5-4399-a144-e6a051b5820c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.551469 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ff7fa-5604-4cab-8bbb-1b9153f5c884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.551976 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ff7fa-5604-4cab-8bbb-1b9153f5c884 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.552383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ff7fa-5604-4cab-8bbb-1b9153f5c884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.552713 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ff7fa-5604-4cab-8bbb-1b9153f5c884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.553352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ff7fa-5604-4cab-8bbb-1b9153f5c884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.554954 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ff7fa-5604-4cab-8bbb-1b9153f5c884 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.563575 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 620/1247] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:17.564429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad977f1-d4e5-4399-a144-e6a051b5820c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.570462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 628/1248] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:17.571682 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ff7fa-5604-4cab-8bbb-1b9153f5c884 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.576139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad977f1-d4e5-4399-a144-e6a051b5820c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.580902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9effe6-4308-4ba9-a9bf-4f59484554db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.581506 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9effe6-4308-4ba9-a9bf-4f59484554db admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.581607 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ff7fa-5604-4cab-8bbb-1b9153f5c884 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.581785 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9effe6-4308-4ba9-a9bf-4f59484554db admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.582056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9effe6-4308-4ba9-a9bf-4f59484554db admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.582696 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9effe6-4308-4ba9-a9bf-4f59484554db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:17.583002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9effe6-4308-4ba9-a9bf-4f59484554db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.585307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9effe6-4308-4ba9-a9bf-4f59484554db admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.587265 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bac38f-b227-4ef3-81a5-f3b9f5f7f44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.587863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bac38f-b227-4ef3-81a5-f3b9f5f7f44f admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.588148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bac38f-b227-4ef3-81a5-f3b9f5f7f44f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.588421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bac38f-b227-4ef3-81a5-f3b9f5f7f44f admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.589073 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bac38f-b227-4ef3-81a5-f3b9f5f7f44f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.591030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bac38f-b227-4ef3-81a5-f3b9f5f7f44f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.598796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 621/1249] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:17.600327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9effe6-4308-4ba9-a9bf-4f59484554db admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.617077 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9effe6-4308-4ba9-a9bf-4f59484554db admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.621817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 629/1250] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:17.622049 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6e734-c2a5-4359-8014-c1956ae0f666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.622660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6e734-c2a5-4359-8014-c1956ae0f666 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.622956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bac38f-b227-4ef3-81a5-f3b9f5f7f44f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.623093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6e734-c2a5-4359-8014-c1956ae0f666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.623277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6e734-c2a5-4359-8014-c1956ae0f666 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.623904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6e734-c2a5-4359-8014-c1956ae0f666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.625833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6e734-c2a5-4359-8014-c1956ae0f666 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.657897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 622/1251] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:17.657897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec525e7-7c45-4910-943f-e040f168b8e0 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.657897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec525e7-7c45-4910-943f-e040f168b8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.657897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec525e7-7c45-4910-943f-e040f168b8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.684643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-eec525e7-7c45-4910-943f-e040f168b8e0 None None] MFA Rules not processed for user `503eb61fecdf4509bb21dd453027ee79`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:17.685861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-eec525e7-7c45-4910-943f-e040f168b8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:17.686155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bac38f-b227-4ef3-81a5-f3b9f5f7f44f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.702723 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:17.746205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77076e37-4e67-4227-8c67-df38870f6b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.746881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77076e37-4e67-4227-8c67-df38870f6b6d admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.747210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77076e37-4e67-4227-8c67-df38870f6b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.747489 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77076e37-4e67-4227-8c67-df38870f6b6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.748125 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77076e37-4e67-4227-8c67-df38870f6b6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.750153 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77076e37-4e67-4227-8c67-df38870f6b6d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.758048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 623/1252] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:17.759013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec525e7-7c45-4910-943f-e040f168b8e0 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.769595 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec525e7-7c45-4910-943f-e040f168b8e0 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.775414 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f781a1-1e61-498e-bdf3-8e5c11611885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.775988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f781a1-1e61-498e-bdf3-8e5c11611885 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.776271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f781a1-1e61-498e-bdf3-8e5c11611885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.776620 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f781a1-1e61-498e-bdf3-8e5c11611885 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.777256 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f781a1-1e61-498e-bdf3-8e5c11611885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.781558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 630/1253] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:17.781558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77076e37-4e67-4227-8c67-df38870f6b6d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.781812 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f781a1-1e61-498e-bdf3-8e5c11611885 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.785142 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:17.807798 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 624/1254] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:17.808889 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f781a1-1e61-498e-bdf3-8e5c11611885 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.845476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77076e37-4e67-4227-8c67-df38870f6b6d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.870320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f781a1-1e61-498e-bdf3-8e5c11611885 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.874892 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e093b91-aaf6-4ca1-92af-3e47c52aafcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.875597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e093b91-aaf6-4ca1-92af-3e47c52aafcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.875874 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e093b91-aaf6-4ca1-92af-3e47c52aafcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.876152 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e093b91-aaf6-4ca1-92af-3e47c52aafcd admin admin] PATH_INFO: `/v3/projects/efb5f401afb04c6ca3fdd2c437174615/users/59b2758b1f7e477aa7db11bb1951497d/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.890721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e093b91-aaf6-4ca1-92af-3e47c52aafcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb5f401afb04c6ca3fdd2c437174615, user_id=59b2758b1f7e477aa7db11bb1951497d, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.893410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e093b91-aaf6-4ca1-92af-3e47c52aafcd admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.901984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3779f1-eaf0-4579-8fad-8a64ab5b9da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:17.902602 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3779f1-eaf0-4579-8fad-8a64ab5b9da1 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:17.902886 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3779f1-eaf0-4579-8fad-8a64ab5b9da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:17.903188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3779f1-eaf0-4579-8fad-8a64ab5b9da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:17.903821 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3779f1-eaf0-4579-8fad-8a64ab5b9da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:17.905709 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3779f1-eaf0-4579-8fad-8a64ab5b9da1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:17.907269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 625/1255] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:17 2022] PUT /identity/v3/projects/efb5f401afb04c6ca3fdd2c437174615/users/59b2758b1f7e477aa7db11bb1951497d/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:17.908148 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e093b91-aaf6-4ca1-92af-3e47c52aafcd admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.938470 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 631/1256] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:17.938470 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3779f1-eaf0-4579-8fad-8a64ab5b9da1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:17.973820 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e093b91-aaf6-4ca1-92af-3e47c52aafcd admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:17.980352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:18.001979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3779f1-eaf0-4579-8fad-8a64ab5b9da1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.007671 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1a275-5ba4-44fc-be00-be3eb995041f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.008268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1a275-5ba4-44fc-be00-be3eb995041f admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.008482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1a275-5ba4-44fc-be00-be3eb995041f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.008749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1a275-5ba4-44fc-be00-be3eb995041f admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.009371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1a275-5ba4-44fc-be00-be3eb995041f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.011484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1a275-5ba4-44fc-be00-be3eb995041f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.035179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c4058-e87c-49e6-ada6-31f6383b0594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.035179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c4058-e87c-49e6-ada6-31f6383b0594 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.035179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c4058-e87c-49e6-ada6-31f6383b0594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.035179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c4058-e87c-49e6-ada6-31f6383b0594 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.035758 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c4058-e87c-49e6-ada6-31f6383b0594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.036711 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 632/1257] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:18.037717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1a275-5ba4-44fc-be00-be3eb995041f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.037863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c4058-e87c-49e6-ada6-31f6383b0594 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.047156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1a275-5ba4-44fc-be00-be3eb995041f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.051775 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89793f67-e164-40f9-bc1f-fa3b7d531f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.053140 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89793f67-e164-40f9-bc1f-fa3b7d531f5a admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.053540 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89793f67-e164-40f9-bc1f-fa3b7d531f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.053820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89793f67-e164-40f9-bc1f-fa3b7d531f5a admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.054445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89793f67-e164-40f9-bc1f-fa3b7d531f5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.056628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89793f67-e164-40f9-bc1f-fa3b7d531f5a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.065978 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 626/1258] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:17 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:18.066876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c4058-e87c-49e6-ada6-31f6383b0594 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.072334 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:18.084191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 633/1259] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:18 2022] POST /identity/v3/users => generated 294 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:18.085330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89793f67-e164-40f9-bc1f-fa3b7d531f5a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.094752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89793f67-e164-40f9-bc1f-fa3b7d531f5a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.137950 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c4058-e87c-49e6-ada6-31f6383b0594 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.142773 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2dbf9-1146-4efd-979c-756c98173ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.143422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2dbf9-1146-4efd-979c-756c98173ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.143699 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2dbf9-1146-4efd-979c-756c98173ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.143973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2dbf9-1146-4efd-979c-756c98173ad5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.144601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2dbf9-1146-4efd-979c-756c98173ad5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.147085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2dbf9-1146-4efd-979c-756c98173ad5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.153176 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e69a8-76cf-4541-96f5-27ed3bb2978c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.153797 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e69a8-76cf-4541-96f5-27ed3bb2978c admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.154079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e69a8-76cf-4541-96f5-27ed3bb2978c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.154352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e69a8-76cf-4541-96f5-27ed3bb2978c admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.154977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e69a8-76cf-4541-96f5-27ed3bb2978c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.156986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e69a8-76cf-4541-96f5-27ed3bb2978c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.177964 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 627/1260] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:18 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:18.177964 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2dbf9-1146-4efd-979c-756c98173ad5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.180817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 634/1261] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:18 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:18.182004 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96e75c-495c-4d63-8296-9c49bb461101 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.182312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96e75c-495c-4d63-8296-9c49bb461101 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.182579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96e75c-495c-4d63-8296-9c49bb461101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.210284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4e96e75c-495c-4d63-8296-9c49bb461101 None None] MFA Rules not processed for user `59b2758b1f7e477aa7db11bb1951497d`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:18.211533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4e96e75c-495c-4d63-8296-9c49bb461101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:18.239712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2dbf9-1146-4efd-979c-756c98173ad5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.244789 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225a1fc-bb2e-4235-9534-27c3d462cb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.245393 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225a1fc-bb2e-4235-9534-27c3d462cb6d admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.245671 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225a1fc-bb2e-4235-9534-27c3d462cb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.245944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225a1fc-bb2e-4235-9534-27c3d462cb6d admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.246565 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225a1fc-bb2e-4235-9534-27c3d462cb6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.247597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-c225a1fc-bb2e-4235-9534-27c3d462cb6d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:24:18.247597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 14 02:24:18.247597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: Oct 14 02:24:18.248801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225a1fc-bb2e-4235-9534-27c3d462cb6d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.270590 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 628/1262] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:18 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:18.272410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225a1fc-bb2e-4235-9534-27c3d462cb6d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.279325 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 635/1263] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:18.282546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225a1fc-bb2e-4235-9534-27c3d462cb6d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.283463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96e75c-495c-4d63-8296-9c49bb461101 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.287950 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf3588-3fdc-4f9e-9a8b-08e3da35a09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.288497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3588-3fdc-4f9e-9a8b-08e3da35a09b admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.288777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3588-3fdc-4f9e-9a8b-08e3da35a09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.289241 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3588-3fdc-4f9e-9a8b-08e3da35a09b admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.289861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf3588-3fdc-4f9e-9a8b-08e3da35a09b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.292368 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf3588-3fdc-4f9e-9a8b-08e3da35a09b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.294022 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96e75c-495c-4d63-8296-9c49bb461101 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.298650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161e087-b9f5-40b9-8adf-30e9b14cd4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.299349 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161e087-b9f5-40b9-8adf-30e9b14cd4bb admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.299648 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161e087-b9f5-40b9-8adf-30e9b14cd4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.299924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161e087-b9f5-40b9-8adf-30e9b14cd4bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.300581 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161e087-b9f5-40b9-8adf-30e9b14cd4bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.302209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161e087-b9f5-40b9-8adf-30e9b14cd4bb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.312437 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 636/1264] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:18.316256 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 629/1265] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:18 2022] POST /identity/v3/users => generated 290 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:18.316404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161e087-b9f5-40b9-8adf-30e9b14cd4bb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.316509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf3588-3fdc-4f9e-9a8b-08e3da35a09b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.325877 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161e087-b9f5-40b9-8adf-30e9b14cd4bb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.326111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf3588-3fdc-4f9e-9a8b-08e3da35a09b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.330454 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2c1ba-d949-46ff-a022-cd5c4ddf85a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.330642 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03645ddc-7e65-4483-b017-7223de9f4ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.331207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2c1ba-d949-46ff-a022-cd5c4ddf85a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.331313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03645ddc-7e65-4483-b017-7223de9f4ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.331491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2c1ba-d949-46ff-a022-cd5c4ddf85a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.331601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03645ddc-7e65-4483-b017-7223de9f4ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.331767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2c1ba-d949-46ff-a022-cd5c4ddf85a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.331889 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03645ddc-7e65-4483-b017-7223de9f4ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.332424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2c1ba-d949-46ff-a022-cd5c4ddf85a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.332526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03645ddc-7e65-4483-b017-7223de9f4ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.334685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03645ddc-7e65-4483-b017-7223de9f4ac2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.334848 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2c1ba-d949-46ff-a022-cd5c4ddf85a8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.345207 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 630/1266] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:18.345956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03645ddc-7e65-4483-b017-7223de9f4ac2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.353749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:18.356506 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03645ddc-7e65-4483-b017-7223de9f4ac2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.362227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 637/1267] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:18 2022] POST /identity/v3/users => generated 292 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:18.362227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2c1ba-d949-46ff-a022-cd5c4ddf85a8 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.362489 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7469b95-a9d1-4425-9451-c8d42df0eed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.362489 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7469b95-a9d1-4425-9451-c8d42df0eed9 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.362750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7469b95-a9d1-4425-9451-c8d42df0eed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.363055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7469b95-a9d1-4425-9451-c8d42df0eed9 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.363683 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7469b95-a9d1-4425-9451-c8d42df0eed9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.366393 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7469b95-a9d1-4425-9451-c8d42df0eed9 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.371950 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2c1ba-d949-46ff-a022-cd5c4ddf85a8 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.376553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b96c43-ec79-4598-a84b-4f2b6009fdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.377151 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b96c43-ec79-4598-a84b-4f2b6009fdab admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.377438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b96c43-ec79-4598-a84b-4f2b6009fdab admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.377717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b96c43-ec79-4598-a84b-4f2b6009fdab admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.378371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b96c43-ec79-4598-a84b-4f2b6009fdab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.380420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b96c43-ec79-4598-a84b-4f2b6009fdab admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.388323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 631/1268] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:18 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:18.389214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7469b95-a9d1-4425-9451-c8d42df0eed9 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.393061 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 638/1269] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:18.393859 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b96c43-ec79-4598-a84b-4f2b6009fdab admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.401533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7469b95-a9d1-4425-9451-c8d42df0eed9 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.403570 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b96c43-ec79-4598-a84b-4f2b6009fdab admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.406799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcefed-d56d-4265-bc77-09f3837d17b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.407455 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcefed-d56d-4265-bc77-09f3837d17b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.407735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcefed-d56d-4265-bc77-09f3837d17b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.408012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcefed-d56d-4265-bc77-09f3837d17b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.408175 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22eb20-a593-4205-847c-b1f2b54eb498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.408650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22eb20-a593-4205-847c-b1f2b54eb498 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.408777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcefed-d56d-4265-bc77-09f3837d17b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.408931 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22eb20-a593-4205-847c-b1f2b54eb498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.409208 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22eb20-a593-4205-847c-b1f2b54eb498 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.409866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22eb20-a593-4205-847c-b1f2b54eb498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.410264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcefed-d56d-4265-bc77-09f3837d17b2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.412592 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22eb20-a593-4205-847c-b1f2b54eb498 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.421386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 632/1270] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:18.422199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcefed-d56d-4265-bc77-09f3837d17b2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.431747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcefed-d56d-4265-bc77-09f3837d17b2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.436711 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8452b-2ca5-4171-a8a7-2ae39595ca7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.437313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8452b-2ca5-4171-a8a7-2ae39595ca7e admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.437591 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8452b-2ca5-4171-a8a7-2ae39595ca7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.437865 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8452b-2ca5-4171-a8a7-2ae39595ca7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.438492 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8452b-2ca5-4171-a8a7-2ae39595ca7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.440943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8452b-2ca5-4171-a8a7-2ae39595ca7e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.442091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 639/1271] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:18 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:18.443833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22eb20-a593-4205-847c-b1f2b54eb498 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.453650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 633/1272] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:18.454576 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8452b-2ca5-4171-a8a7-2ae39595ca7e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.463997 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8452b-2ca5-4171-a8a7-2ae39595ca7e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.468954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:18.512702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22eb20-a593-4205-847c-b1f2b54eb498 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.517397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234374cd-6c53-4f3f-8a45-fd876685b393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.517992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234374cd-6c53-4f3f-8a45-fd876685b393 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.518271 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234374cd-6c53-4f3f-8a45-fd876685b393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.518543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234374cd-6c53-4f3f-8a45-fd876685b393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.519222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234374cd-6c53-4f3f-8a45-fd876685b393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.520891 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234374cd-6c53-4f3f-8a45-fd876685b393 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.527411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b76de-b121-4925-80d7-b9cc5d755ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.528049 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b76de-b121-4925-80d7-b9cc5d755ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.528321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b76de-b121-4925-80d7-b9cc5d755ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.528589 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b76de-b121-4925-80d7-b9cc5d755ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.529217 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b76de-b121-4925-80d7-b9cc5d755ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.530799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b76de-b121-4925-80d7-b9cc5d755ca3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.531343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 640/1273] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:18.532183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234374cd-6c53-4f3f-8a45-fd876685b393 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.542440 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234374cd-6c53-4f3f-8a45-fd876685b393 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.544153 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 634/1274] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:18.545127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b76de-b121-4925-80d7-b9cc5d755ca3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.547171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b9043-f935-4df3-9126-90c648ff0b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.547593 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b9043-f935-4df3-9126-90c648ff0b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.547873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b9043-f935-4df3-9126-90c648ff0b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.548149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b9043-f935-4df3-9126-90c648ff0b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.548789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b9043-f935-4df3-9126-90c648ff0b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.550408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b9043-f935-4df3-9126-90c648ff0b01 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.555835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b76de-b121-4925-80d7-b9cc5d755ca3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.560629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6e427-2582-476f-b3de-9ed21b7c52f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.561227 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6e427-2582-476f-b3de-9ed21b7c52f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.564432 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 641/1275] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:18.564904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6e427-2582-476f-b3de-9ed21b7c52f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.564904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6e427-2582-476f-b3de-9ed21b7c52f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.565070 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b9043-f935-4df3-9126-90c648ff0b01 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.565183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6e427-2582-476f-b3de-9ed21b7c52f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.565183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6e427-2582-476f-b3de-9ed21b7c52f4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.572022 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b9043-f935-4df3-9126-90c648ff0b01 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.575107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 635/1276] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:18.575273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6e427-2582-476f-b3de-9ed21b7c52f4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.579784 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e3ffa-c181-4dd4-873a-a02956eb9b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.580404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e3ffa-c181-4dd4-873a-a02956eb9b96 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.580669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e3ffa-c181-4dd4-873a-a02956eb9b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.580944 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e3ffa-c181-4dd4-873a-a02956eb9b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.581584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e3ffa-c181-4dd4-873a-a02956eb9b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.583230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e3ffa-c181-4dd4-873a-a02956eb9b96 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.585161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6e427-2582-476f-b3de-9ed21b7c52f4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.589851 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14ebe6-8475-4ddd-864b-7b750622686a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.590431 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14ebe6-8475-4ddd-864b-7b750622686a admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.590728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14ebe6-8475-4ddd-864b-7b750622686a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.590977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14ebe6-8475-4ddd-864b-7b750622686a admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.591633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14ebe6-8475-4ddd-864b-7b750622686a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.593188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14ebe6-8475-4ddd-864b-7b750622686a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.593307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 642/1277] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:18.593969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e3ffa-c181-4dd4-873a-a02956eb9b96 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.604049 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e3ffa-c181-4dd4-873a-a02956eb9b96 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.604502 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 636/1278] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:18.605354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14ebe6-8475-4ddd-864b-7b750622686a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.611970 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad076c8f-4e85-4b72-ba03-bf277314aa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.612577 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad076c8f-4e85-4b72-ba03-bf277314aa01 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.612846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad076c8f-4e85-4b72-ba03-bf277314aa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.613119 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad076c8f-4e85-4b72-ba03-bf277314aa01 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.613739 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad076c8f-4e85-4b72-ba03-bf277314aa01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.614822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14ebe6-8475-4ddd-864b-7b750622686a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.616374 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad076c8f-4e85-4b72-ba03-bf277314aa01 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.619645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab31d7-75b0-4993-b235-dc23b34c7813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.620303 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab31d7-75b0-4993-b235-dc23b34c7813 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.620576 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab31d7-75b0-4993-b235-dc23b34c7813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.620844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab31d7-75b0-4993-b235-dc23b34c7813 admin admin] PATH_INFO: `/v3/projects/e81bc6b4efe24f1da9b1d75c335bb73e/users/fbea6f3499804746b0daa91860584f43/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.635915 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab31d7-75b0-4993-b235-dc23b34c7813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81bc6b4efe24f1da9b1d75c335bb73e, user_id=fbea6f3499804746b0daa91860584f43, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.638565 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab31d7-75b0-4993-b235-dc23b34c7813 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.638698 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 643/1279] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:18 2022] POST /identity/v3/users => generated 285 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:18.639545 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad076c8f-4e85-4b72-ba03-bf277314aa01 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.650463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad076c8f-4e85-4b72-ba03-bf277314aa01 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.651520 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 637/1280] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:18 2022] PUT /identity/v3/projects/e81bc6b4efe24f1da9b1d75c335bb73e/users/fbea6f3499804746b0daa91860584f43/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:18.652301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab31d7-75b0-4993-b235-dc23b34c7813 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.706816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b319a9e-c555-473f-8ac1-448783d8b2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.707451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b319a9e-c555-473f-8ac1-448783d8b2f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.707730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b319a9e-c555-473f-8ac1-448783d8b2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.708018 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b319a9e-c555-473f-8ac1-448783d8b2f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.708686 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b319a9e-c555-473f-8ac1-448783d8b2f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.710270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b319a9e-c555-473f-8ac1-448783d8b2f7 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.718278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab31d7-75b0-4993-b235-dc23b34c7813 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.723048 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 644/1281] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:18.723048 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b319a9e-c555-473f-8ac1-448783d8b2f7 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.727939 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399eb3f9-b0af-40c7-8201-53f53f1c5823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.728818 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eb3f9-b0af-40c7-8201-53f53f1c5823 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.729089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eb3f9-b0af-40c7-8201-53f53f1c5823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.729371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eb3f9-b0af-40c7-8201-53f53f1c5823 admin admin] PATH_INFO: `/v3/projects/e0d86bdb69fa4c6ca37a301b8c57ec5e/users/e4ebab767c054b289e2279659a1cf2e1/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.731159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b319a9e-c555-473f-8ac1-448783d8b2f7 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.739103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2da32d-ff83-4fa9-98f1-a2aa9bd5f31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.739103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2da32d-ff83-4fa9-98f1-a2aa9bd5f31f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.739103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2da32d-ff83-4fa9-98f1-a2aa9bd5f31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.739103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2da32d-ff83-4fa9-98f1-a2aa9bd5f31f admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.739542 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2da32d-ff83-4fa9-98f1-a2aa9bd5f31f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.739542 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2da32d-ff83-4fa9-98f1-a2aa9bd5f31f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.745132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399eb3f9-b0af-40c7-8201-53f53f1c5823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0d86bdb69fa4c6ca37a301b8c57ec5e, user_id=e4ebab767c054b289e2279659a1cf2e1, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.747843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399eb3f9-b0af-40c7-8201-53f53f1c5823 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.750603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 645/1282] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:18.751422 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2da32d-ff83-4fa9-98f1-a2aa9bd5f31f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.761429 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2da32d-ff83-4fa9-98f1-a2aa9bd5f31f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.764928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 638/1283] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:18 2022] PUT /identity/v3/projects/e0d86bdb69fa4c6ca37a301b8c57ec5e/users/e4ebab767c054b289e2279659a1cf2e1/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:18.765722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399eb3f9-b0af-40c7-8201-53f53f1c5823 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.766755 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:18.813444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:18.824359 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda8867-75c3-45dc-a202-b314241d7893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.825889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda8867-75c3-45dc-a202-b314241d7893 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.826214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda8867-75c3-45dc-a202-b314241d7893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.826692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda8867-75c3-45dc-a202-b314241d7893 admin admin] PATH_INFO: `/v3/projects/dcbfe035b11a424a827cb5522e6ef8f2/users/7cebd05c17824d478e57138e521a5e11/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.832279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399eb3f9-b0af-40c7-8201-53f53f1c5823 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.837426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a2611-8411-4821-930d-044ade2d6dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.838321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a2611-8411-4821-930d-044ade2d6dd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.838801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a2611-8411-4821-930d-044ade2d6dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.839042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a2611-8411-4821-930d-044ade2d6dd2 admin admin] PATH_INFO: `/v3/projects/b13c82f939b04b2bb2e370d5de09fe3f/users/d588355d921e4029bfa7ca4087f36c2c/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.845345 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda8867-75c3-45dc-a202-b314241d7893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcbfe035b11a424a827cb5522e6ef8f2, user_id=7cebd05c17824d478e57138e521a5e11, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.848267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda8867-75c3-45dc-a202-b314241d7893 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.856071 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a2611-8411-4821-930d-044ade2d6dd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b13c82f939b04b2bb2e370d5de09fe3f, user_id=d588355d921e4029bfa7ca4087f36c2c, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.858836 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a2611-8411-4821-930d-044ade2d6dd2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.861934 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 646/1284] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:18 2022] PUT /identity/v3/projects/dcbfe035b11a424a827cb5522e6ef8f2/users/7cebd05c17824d478e57138e521a5e11/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:18.862692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda8867-75c3-45dc-a202-b314241d7893 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.875672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 639/1285] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:18 2022] PUT /identity/v3/projects/b13c82f939b04b2bb2e370d5de09fe3f/users/d588355d921e4029bfa7ca4087f36c2c/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:18.876588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a2611-8411-4821-930d-044ade2d6dd2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.935777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda8867-75c3-45dc-a202-b314241d7893 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.940298 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac5962-0d09-4ae4-9b82-22ed6e6ae24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.940959 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac5962-0d09-4ae4-9b82-22ed6e6ae24c admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.941234 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac5962-0d09-4ae4-9b82-22ed6e6ae24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.941506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac5962-0d09-4ae4-9b82-22ed6e6ae24c admin admin] PATH_INFO: `/v3/projects/a2e38a12e174437982573067f6bc3f8d/users/79365a5e4ae2431d91a4aa603ba58fee/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.944958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a2611-8411-4821-930d-044ade2d6dd2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:18.949659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d8a58-7b39-4108-b448-52c5ff1b6788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:18.950262 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d8a58-7b39-4108-b448-52c5ff1b6788 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.950549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d8a58-7b39-4108-b448-52c5ff1b6788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.950826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d8a58-7b39-4108-b448-52c5ff1b6788 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.951502 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d8a58-7b39-4108-b448-52c5ff1b6788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.953105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d8a58-7b39-4108-b448-52c5ff1b6788 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.956832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac5962-0d09-4ae4-9b82-22ed6e6ae24c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e38a12e174437982573067f6bc3f8d, user_id=79365a5e4ae2431d91a4aa603ba58fee, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:18.960246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac5962-0d09-4ae4-9b82-22ed6e6ae24c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:18.966331 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 640/1286] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:18.966331 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8c17d-dcc1-417a-a70b-bf2a57b96494 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:18.966331 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8c17d-dcc1-417a-a70b-bf2a57b96494 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:18.966331 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8c17d-dcc1-417a-a70b-bf2a57b96494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:18.978419 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 647/1287] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:18 2022] PUT /identity/v3/projects/a2e38a12e174437982573067f6bc3f8d/users/79365a5e4ae2431d91a4aa603ba58fee/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:18.978996 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac5962-0d09-4ae4-9b82-22ed6e6ae24c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:18.998729 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-8dd8c17d-dcc1-417a-a70b-bf2a57b96494 None None] MFA Rules not processed for user `fbea6f3499804746b0daa91860584f43`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:18.999992 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8dd8c17d-dcc1-417a-a70b-bf2a57b96494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:19.041787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac5962-0d09-4ae4-9b82-22ed6e6ae24c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:19.046855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260e906-b705-4181-8ce2-65c8b76a87ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:19.047548 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260e906-b705-4181-8ce2-65c8b76a87ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.047883 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260e906-b705-4181-8ce2-65c8b76a87ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.048167 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260e906-b705-4181-8ce2-65c8b76a87ca admin admin] PATH_INFO: `/v3/projects/1a37a13b40e54342bb3942ff4387534e/users/bad955c1488c4c3380db7e84403253ee/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.063352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260e906-b705-4181-8ce2-65c8b76a87ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a37a13b40e54342bb3942ff4387534e, user_id=bad955c1488c4c3380db7e84403253ee, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:19.069492 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 641/1288] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:19.069492 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8c17d-dcc1-417a-a70b-bf2a57b96494 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:19.069754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260e906-b705-4181-8ce2-65c8b76a87ca admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:19.076816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8c17d-dcc1-417a-a70b-bf2a57b96494 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:19.077560 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:19.091425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 648/1289] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:18 2022] PUT /identity/v3/projects/1a37a13b40e54342bb3942ff4387534e/users/bad955c1488c4c3380db7e84403253ee/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:19.091425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f048297-7908-469e-93f8-ffb482889585 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.091425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f048297-7908-469e-93f8-ffb482889585 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.091425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f048297-7908-469e-93f8-ffb482889585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.109571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0f048297-7908-469e-93f8-ffb482889585 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:19.110886 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0f048297-7908-469e-93f8-ffb482889585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:19.132662 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:19.144251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ae71e-d9e3-4e73-b860-59d5dc964714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:19.144953 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ae71e-d9e3-4e73-b860-59d5dc964714 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.145238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ae71e-d9e3-4e73-b860-59d5dc964714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.145534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ae71e-d9e3-4e73-b860-59d5dc964714 admin admin] PATH_INFO: `/v3/projects/1d2f70db22ec43d5ba9125f940433320/users/7d4748cbe63d490b9d8b6914a27ff364/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.161780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ae71e-d9e3-4e73-b860-59d5dc964714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2f70db22ec43d5ba9125f940433320, user_id=7d4748cbe63d490b9d8b6914a27ff364, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:19.164528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ae71e-d9e3-4e73-b860-59d5dc964714 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:19.172293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 649/1290] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:19.173286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f048297-7908-469e-93f8-ffb482889585 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:19.181742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 642/1291] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:19 2022] PUT /identity/v3/projects/1d2f70db22ec43d5ba9125f940433320/users/7d4748cbe63d490b9d8b6914a27ff364/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:19.183592 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f87c4-bb0c-4e04-81e9-794dc772f154 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.183946 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f87c4-bb0c-4e04-81e9-794dc772f154 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.184288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f87c4-bb0c-4e04-81e9-794dc772f154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.184572 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f048297-7908-469e-93f8-ffb482889585 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:19.211804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-df4f87c4-bb0c-4e04-81e9-794dc772f154 None None] MFA Rules not processed for user `d588355d921e4029bfa7ca4087f36c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:19.213057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-df4f87c4-bb0c-4e04-81e9-794dc772f154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:19.241940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879095e-14cd-43f1-80ac-dfd7073f7653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:19.242937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879095e-14cd-43f1-80ac-dfd7073f7653 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.243249 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879095e-14cd-43f1-80ac-dfd7073f7653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.243518 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879095e-14cd-43f1-80ac-dfd7073f7653 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.244150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879095e-14cd-43f1-80ac-dfd7073f7653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:19.245728 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879095e-14cd-43f1-80ac-dfd7073f7653 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:19.255251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 650/1292] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:19.256012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879095e-14cd-43f1-80ac-dfd7073f7653 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:19.265587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879095e-14cd-43f1-80ac-dfd7073f7653 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:19.270087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5981f2-7665-428d-980a-3dd968e3ab1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:19.270716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5981f2-7665-428d-980a-3dd968e3ab1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.270993 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5981f2-7665-428d-980a-3dd968e3ab1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.271292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5981f2-7665-428d-980a-3dd968e3ab1e admin admin] PATH_INFO: `/v3/projects/549de84684f646b0a4b237a0461ca160/users/d5d60c72d8944e6bb61f01ddacf5864f/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.281796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 643/1293] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:19.283344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be464b-ce7d-4e15-95ce-20dffa814a45 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.283722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be464b-ce7d-4e15-95ce-20dffa814a45 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.283912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be464b-ce7d-4e15-95ce-20dffa814a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.285377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5981f2-7665-428d-980a-3dd968e3ab1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=549de84684f646b0a4b237a0461ca160, user_id=d5d60c72d8944e6bb61f01ddacf5864f, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:19.288713 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5981f2-7665-428d-980a-3dd968e3ab1e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:19.304568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 651/1294] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:19 2022] PUT /identity/v3/projects/549de84684f646b0a4b237a0461ca160/users/d5d60c72d8944e6bb61f01ddacf5864f/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:19.305941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474e05c-3d90-4668-b34c-c740cb01c39e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.306111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e6be464b-ce7d-4e15-95ce-20dffa814a45 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:19.306252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474e05c-3d90-4668-b34c-c740cb01c39e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.306623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474e05c-3d90-4668-b34c-c740cb01c39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.308509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e6be464b-ce7d-4e15-95ce-20dffa814a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:19.327941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b474e05c-3d90-4668-b34c-c740cb01c39e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:19.329063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b474e05c-3d90-4668-b34c-c740cb01c39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:19.369291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 644/1295] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:19.370681 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bd1ca-1806-4831-81fb-37019691a9d1 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.370967 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bd1ca-1806-4831-81fb-37019691a9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.371272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bd1ca-1806-4831-81fb-37019691a9d1 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.377038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 645/1296] 10.222.0.18 () {54 vars in 961 bytes} [Fri Oct 14 02:24:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:24:19.378352 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b521d17-bfbe-4c99-80ac-1b2cdb09a1bf None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.378629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b521d17-bfbe-4c99-80ac-1b2cdb09a1bf None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.378895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b521d17-bfbe-4c99-80ac-1b2cdb09a1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.394231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 652/1297] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:19.395185 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474e05c-3d90-4668-b34c-c740cb01c39e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:19.399117 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-8b521d17-bfbe-4c99-80ac-1b2cdb09a1bf None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:19.400417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8b521d17-bfbe-4c99-80ac-1b2cdb09a1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:19.407624 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:19.412841 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 646/1298] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:19.412841 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b521d17-bfbe-4c99-80ac-1b2cdb09a1bf None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:19.422329 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b521d17-bfbe-4c99-80ac-1b2cdb09a1bf None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:19.427360 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979c608-a8fc-466e-bc11-73de3e0e871a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:19.428108 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979c608-a8fc-466e-bc11-73de3e0e871a admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.428382 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979c608-a8fc-466e-bc11-73de3e0e871a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.428658 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979c608-a8fc-466e-bc11-73de3e0e871a admin admin] PATH_INFO: `/v3/projects/dcbfe035b11a424a827cb5522e6ef8f2/users/7cebd05c17824d478e57138e521a5e11/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.430547 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979c608-a8fc-466e-bc11-73de3e0e871a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcbfe035b11a424a827cb5522e6ef8f2, user_id=7cebd05c17824d478e57138e521a5e11, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:19.433236 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979c608-a8fc-466e-bc11-73de3e0e871a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:19.445313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 647/1299] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:19 2022] PUT /identity/v3/projects/dcbfe035b11a424a827cb5522e6ef8f2/users/7cebd05c17824d478e57138e521a5e11/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:19.446607 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e08620-cf82-4dda-a950-4274dab58994 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.446896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e08620-cf82-4dda-a950-4274dab58994 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.447197 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e08620-cf82-4dda-a950-4274dab58994 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.448555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 648/1300] 10.222.0.18 () {54 vars in 961 bytes} [Fri Oct 14 02:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:24:19.449330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e08620-cf82-4dda-a950-4274dab58994 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:19.452627 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474e05c-3d90-4668-b34c-c740cb01c39e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:19.497291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:19.502344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de3dac-2598-4d96-a9fb-83cb7558ed98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:19.502980 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de3dac-2598-4d96-a9fb-83cb7558ed98 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.503284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de3dac-2598-4d96-a9fb-83cb7558ed98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.503550 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de3dac-2598-4d96-a9fb-83cb7558ed98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.517138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e08620-cf82-4dda-a950-4274dab58994 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:19.522201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442415a-4479-4278-9c11-1b3513e4b693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:19.522805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442415a-4479-4278-9c11-1b3513e4b693 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.523101 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442415a-4479-4278-9c11-1b3513e4b693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.523380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442415a-4479-4278-9c11-1b3513e4b693 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.524017 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5442415a-4479-4278-9c11-1b3513e4b693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:19.525616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5442415a-4479-4278-9c11-1b3513e4b693 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:19.535490 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 649/1301] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:19.536586 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba96b4-276b-4a02-81ee-95fc74575d70 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.536868 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba96b4-276b-4a02-81ee-95fc74575d70 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.537214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba96b4-276b-4a02-81ee-95fc74575d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.569002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de3dac-2598-4d96-a9fb-83cb7558ed98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:19.570456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de3dac-2598-4d96-a9fb-83cb7558ed98 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:19.577182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-91ba96b4-276b-4a02-81ee-95fc74575d70 None None] MFA Rules not processed for user `de13be55b4034477baf4c029c1a79757`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:19.578452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-91ba96b4-276b-4a02-81ee-95fc74575d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:19.580619 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 653/1302] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:19.581505 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de3dac-2598-4d96-a9fb-83cb7558ed98 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:19.590456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de3dac-2598-4d96-a9fb-83cb7558ed98 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:19.595174 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04f466-cc32-40d0-9b44-7d2c611bac33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:19.595833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04f466-cc32-40d0-9b44-7d2c611bac33 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.596111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04f466-cc32-40d0-9b44-7d2c611bac33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.596760 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04f466-cc32-40d0-9b44-7d2c611bac33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.606921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04f466-cc32-40d0-9b44-7d2c611bac33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:19.608584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04f466-cc32-40d0-9b44-7d2c611bac33 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:19.620475 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 654/1303] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:19.621465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04f466-cc32-40d0-9b44-7d2c611bac33 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:19.629683 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04f466-cc32-40d0-9b44-7d2c611bac33 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:19.634789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d31cf-9e7c-4e6c-91cd-e75806637a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:19.635521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d31cf-9e7c-4e6c-91cd-e75806637a0a service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.636002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d31cf-9e7c-4e6c-91cd-e75806637a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.636535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d31cf-9e7c-4e6c-91cd-e75806637a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.647891 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d31cf-9e7c-4e6c-91cd-e75806637a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:19.649504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d31cf-9e7c-4e6c-91cd-e75806637a0a service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:19.653725 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 650/1304] 10.222.0.18 () {58 vars in 1058 bytes} [Fri Oct 14 02:24:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:19.654557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba96b4-276b-4a02-81ee-95fc74575d70 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:19.663311 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 655/1305] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:19.663997 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba96b4-276b-4a02-81ee-95fc74575d70 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:19.665340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ab5ea-16bc-449b-8b29-0aa7873ac4cb None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.665716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ab5ea-16bc-449b-8b29-0aa7873ac4cb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.666043 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ab5ea-16bc-449b-8b29-0aa7873ac4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.668948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cce350-e28a-4a04-8b8f-d044e20f2eee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:19.669261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cce350-e28a-4a04-8b8f-d044e20f2eee service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.669582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cce350-e28a-4a04-8b8f-d044e20f2eee service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.669785 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cce350-e28a-4a04-8b8f-d044e20f2eee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.680390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cce350-e28a-4a04-8b8f-d044e20f2eee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:19.681819 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cce350-e28a-4a04-8b8f-d044e20f2eee service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:19.692898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 651/1306] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:19.693912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65182925-cb12-41ad-94b1-558e8f0111fc None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.694242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65182925-cb12-41ad-94b1-558e8f0111fc None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.694543 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65182925-cb12-41ad-94b1-558e8f0111fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.700191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-295ab5ea-16bc-449b-8b29-0aa7873ac4cb None None] MFA Rules not processed for user `7cebd05c17824d478e57138e521a5e11`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:19.701581 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-295ab5ea-16bc-449b-8b29-0aa7873ac4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:19.715669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-65182925-cb12-41ad-94b1-558e8f0111fc None None] MFA Rules not processed for user `de13be55b4034477baf4c029c1a79757`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:19.716920 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-65182925-cb12-41ad-94b1-558e8f0111fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:19.730028 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 652/1307] 10.222.0.18 () {58 vars in 1058 bytes} [Fri Oct 14 02:24:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:19.730801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65182925-cb12-41ad-94b1-558e8f0111fc None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:19.742469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65182925-cb12-41ad-94b1-558e8f0111fc None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:19.748484 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c274140-5ba4-41c8-b63f-d38ae9bf1f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:19.749315 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c274140-5ba4-41c8-b63f-d38ae9bf1f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.749584 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c274140-5ba4-41c8-b63f-d38ae9bf1f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.749886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c274140-5ba4-41c8-b63f-d38ae9bf1f09 admin admin] PATH_INFO: `/v3/projects/549de84684f646b0a4b237a0461ca160/users/d5d60c72d8944e6bb61f01ddacf5864f/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.752309 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c274140-5ba4-41c8-b63f-d38ae9bf1f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=549de84684f646b0a4b237a0461ca160, user_id=d5d60c72d8944e6bb61f01ddacf5864f, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:19.756369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c274140-5ba4-41c8-b63f-d38ae9bf1f09 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:19.766387 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:19.769537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 653/1308] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:19 2022] PUT /identity/v3/projects/549de84684f646b0a4b237a0461ca160/users/d5d60c72d8944e6bb61f01ddacf5864f/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:19.769537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c274140-5ba4-41c8-b63f-d38ae9bf1f09 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:19.816103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 656/1309] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:19.821899 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ab5ea-16bc-449b-8b29-0aa7873ac4cb None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:19.839769 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ab5ea-16bc-449b-8b29-0aa7873ac4cb None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:19.868503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c274140-5ba4-41c8-b63f-d38ae9bf1f09 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:19.875278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73deec-f847-4f28-b869-983b5c3b6bdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:19.875904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73deec-f847-4f28-b869-983b5c3b6bdf service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.876179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73deec-f847-4f28-b869-983b5c3b6bdf service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.876451 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73deec-f847-4f28-b869-983b5c3b6bdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.901227 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:19.923672 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2eee81-0849-4e35-bb83-9a2016454992 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:19.924438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2eee81-0849-4e35-bb83-9a2016454992 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.924708 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2eee81-0849-4e35-bb83-9a2016454992 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.925013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2eee81-0849-4e35-bb83-9a2016454992 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.949007 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73deec-f847-4f28-b869-983b5c3b6bdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:19.950792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73deec-f847-4f28-b869-983b5c3b6bdf service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:19.976863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 654/1310] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:19.986168 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f16a2-7ff6-4ce2-bcfd-a292d4bcf7bc None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:19.987105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f16a2-7ff6-4ce2-bcfd-a292d4bcf7bc None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:19.987474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f16a2-7ff6-4ce2-bcfd-a292d4bcf7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:19.996789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2eee81-0849-4e35-bb83-9a2016454992 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:19.998850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2eee81-0849-4e35-bb83-9a2016454992 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:20.012844 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 657/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:20.013881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2eee81-0849-4e35-bb83-9a2016454992 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:20.083186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-700f16a2-7ff6-4ce2-bcfd-a292d4bcf7bc None None] MFA Rules not processed for user `d5d60c72d8944e6bb61f01ddacf5864f`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:20.085813 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-700f16a2-7ff6-4ce2-bcfd-a292d4bcf7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:20.102976 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2eee81-0849-4e35-bb83-9a2016454992 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:20.108255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93d76d-f7ea-45cb-b4a9-33ab8e4702ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:20.109411 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93d76d-f7ea-45cb-b4a9-33ab8e4702ed service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:20.110477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93d76d-f7ea-45cb-b4a9-33ab8e4702ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:20.111091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93d76d-f7ea-45cb-b4a9-33ab8e4702ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:20.151203 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:20.220276 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93d76d-f7ea-45cb-b4a9-33ab8e4702ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:20.222532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93d76d-f7ea-45cb-b4a9-33ab8e4702ed service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:20.236750 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 658/1312] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:20.238500 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22eaae-69b3-4381-a01c-602d562aafc7 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:20.238959 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22eaae-69b3-4381-a01c-602d562aafc7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:20.239510 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22eaae-69b3-4381-a01c-602d562aafc7 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:20.242283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 659/1313] 10.222.0.18 () {54 vars in 912 bytes} [Fri Oct 14 02:24:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:24:20.254892 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ef3e9-ca99-4878-844d-b80cf2e7377f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:20.255410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ef3e9-ca99-4878-844d-b80cf2e7377f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:20.255906 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ef3e9-ca99-4878-844d-b80cf2e7377f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:20.273286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 655/1314] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:20.274279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f16a2-7ff6-4ce2-bcfd-a292d4bcf7bc None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:20.284484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-566ef3e9-ca99-4878-844d-b80cf2e7377f None None] MFA Rules not processed for user `de13be55b4034477baf4c029c1a79757`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:20.285907 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-566ef3e9-ca99-4878-844d-b80cf2e7377f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:20.286925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f16a2-7ff6-4ce2-bcfd-a292d4bcf7bc None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:20.293501 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cb3fc-d737-4949-82a4-099bffdd9571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:20.294140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cb3fc-d737-4949-82a4-099bffdd9571 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:20.295792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cb3fc-d737-4949-82a4-099bffdd9571 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:20.296177 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cb3fc-d737-4949-82a4-099bffdd9571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:20.306673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 660/1315] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:24:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:24:20.308024 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1cb3fc-d737-4949-82a4-099bffdd9571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:20.309462 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1cb3fc-d737-4949-82a4-099bffdd9571 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:20.320611 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 656/1316] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:20 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:20.361893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ef3e9-ca99-4878-844d-b80cf2e7377f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:20.379582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ef3e9-ca99-4878-844d-b80cf2e7377f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:20.389073 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc25c55-7544-4033-96aa-b6600acb1f34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:20.390528 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc25c55-7544-4033-96aa-b6600acb1f34 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:20.391385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc25c55-7544-4033-96aa-b6600acb1f34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:20.392492 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc25c55-7544-4033-96aa-b6600acb1f34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:20.416440 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc25c55-7544-4033-96aa-b6600acb1f34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:20.432853 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc25c55-7544-4033-96aa-b6600acb1f34 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:20.472552 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 661/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:20.542985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cb3fc-d737-4949-82a4-099bffdd9571 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:20.562028 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cb3fc-d737-4949-82a4-099bffdd9571 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:20.577059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf60761-a840-46e3-8755-429f0f051d26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:20.577767 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf60761-a840-46e3-8755-429f0f051d26 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:20.578151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf60761-a840-46e3-8755-429f0f051d26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:20.578523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf60761-a840-46e3-8755-429f0f051d26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:20.607938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544eaca6-c464-42ca-b6cd-dcad18c15c64 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:20.608462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544eaca6-c464-42ca-b6cd-dcad18c15c64 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:20.608606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544eaca6-c464-42ca-b6cd-dcad18c15c64 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:20.610311 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 662/1318] 10.222.0.18 () {54 vars in 912 bytes} [Fri Oct 14 02:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:24:20.616596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693dc2f-23db-4f36-a540-6a263a3d4d8e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:20.617011 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693dc2f-23db-4f36-a540-6a263a3d4d8e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:20.617373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693dc2f-23db-4f36-a540-6a263a3d4d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:20.672135 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3693dc2f-23db-4f36-a540-6a263a3d4d8e None None] MFA Rules not processed for user `de13be55b4034477baf4c029c1a79757`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:20.672135 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3693dc2f-23db-4f36-a540-6a263a3d4d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:20.684343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 663/1319] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:24:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:20.726375 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf60761-a840-46e3-8755-429f0f051d26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:20.727907 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf60761-a840-46e3-8755-429f0f051d26 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:20.761777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 657/1320] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:21.131713 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693dc2f-23db-4f36-a540-6a263a3d4d8e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:21.142350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693dc2f-23db-4f36-a540-6a263a3d4d8e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:21.156535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dec2dd-e314-4396-9337-3a5d1fecb05f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:21.157175 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dec2dd-e314-4396-9337-3a5d1fecb05f service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:21.157449 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dec2dd-e314-4396-9337-3a5d1fecb05f service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:21.157717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dec2dd-e314-4396-9337-3a5d1fecb05f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:21.178246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dec2dd-e314-4396-9337-3a5d1fecb05f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:21.181015 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dec2dd-e314-4396-9337-3a5d1fecb05f service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:21.197212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 664/1321] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:24:21.718493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf60761-a840-46e3-8755-429f0f051d26 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:21.807767 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:21.867744 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf60761-a840-46e3-8755-429f0f051d26 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:21.879419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59f5db-4530-414a-8376-7b0d899141cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:21.882270 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59f5db-4530-414a-8376-7b0d899141cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:21.883623 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59f5db-4530-414a-8376-7b0d899141cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:21.884589 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59f5db-4530-414a-8376-7b0d899141cb admin admin] PATH_INFO: `/v3/users/d588355d921e4029bfa7ca4087f36c2c` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:21.887765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59f5db-4530-414a-8376-7b0d899141cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d588355d921e4029bfa7ca4087f36c2c)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:21.892584 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59f5db-4530-414a-8376-7b0d899141cb admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:22.041161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6d59f5db-4530-414a-8376-7b0d899141cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd588355d921e4029bfa7ca4087f36c2c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:22.061050 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6d59f5db-4530-414a-8376-7b0d899141cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd588355d921e4029bfa7ca4087f36c2c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:22.062191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6d59f5db-4530-414a-8376-7b0d899141cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd588355d921e4029bfa7ca4087f36c2c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:22.117620 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6d59f5db-4530-414a-8376-7b0d899141cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd588355d921e4029bfa7ca4087f36c2c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:22.143737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6d59f5db-4530-414a-8376-7b0d899141cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd588355d921e4029bfa7ca4087f36c2c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:22.162024 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 658/1322] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:21 2022] DELETE /identity/v3/users/d588355d921e4029bfa7ca4087f36c2c => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:22.165453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0238-59ec-40b0-8181-cefb7c0289f2 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:22.165951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0238-59ec-40b0-8181-cefb7c0289f2 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:22.166197 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0238-59ec-40b0-8181-cefb7c0289f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:22.203250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-304f0238-59ec-40b0-8181-cefb7c0289f2 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:22.204638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-304f0238-59ec-40b0-8181-cefb7c0289f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:22.344971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 665/1323] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:22.537496 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59f5db-4530-414a-8376-7b0d899141cb admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:22.538945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6d59f5db-4530-414a-8376-7b0d899141cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:22.686931 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59f5db-4530-414a-8376-7b0d899141cb admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:22.695135 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76476c5d-ef83-457d-8395-4303b9826f95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:22.696158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76476c5d-ef83-457d-8395-4303b9826f95 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:22.696309 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76476c5d-ef83-457d-8395-4303b9826f95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:22.696696 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76476c5d-ef83-457d-8395-4303b9826f95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:22.707568 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76476c5d-ef83-457d-8395-4303b9826f95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:22.711208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76476c5d-ef83-457d-8395-4303b9826f95 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:22.726223 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 659/1324] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:22.852251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f0238-59ec-40b0-8181-cefb7c0289f2 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:22.853941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-304f0238-59ec-40b0-8181-cefb7c0289f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:22.876951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f0238-59ec-40b0-8181-cefb7c0289f2 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:22.890544 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:22.892554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:22.893498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:22.894359 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] PATH_INFO: `/v3/users/503eb61fecdf4509bb21dd453027ee79` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:22.898087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=503eb61fecdf4509bb21dd453027ee79)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:22.900183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:24:22.900183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 14 02:24:22.900183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: Oct 14 02:24:22.904023 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:23.072599 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '503eb61fecdf4509bb21dd453027ee79'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:23.091100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '503eb61fecdf4509bb21dd453027ee79'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:23.131901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '503eb61fecdf4509bb21dd453027ee79'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:23.154119 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:23.162852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '503eb61fecdf4509bb21dd453027ee79'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:23.165662 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '503eb61fecdf4509bb21dd453027ee79'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:23.188009 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 666/1325] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:22 2022] DELETE /identity/v3/users/503eb61fecdf4509bb21dd453027ee79 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:23.192851 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c2bb3-9723-4c6b-a384-7ed6fa08f59a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:23.193414 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c2bb3-9723-4c6b-a384-7ed6fa08f59a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:23.193627 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c2bb3-9723-4c6b-a384-7ed6fa08f59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:23.214743 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-6d6c2bb3-9723-4c6b-a384-7ed6fa08f59a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:23.216359 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6d6c2bb3-9723-4c6b-a384-7ed6fa08f59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:23.275812 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:23.299172 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 660/1326] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:23.331377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:23.337484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:23.469525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8361a6-94a4-443b-8500-0b3ca10665dc admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:23.475651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bc10b-77c4-42e4-9c01-ed1fbe877339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:23.476873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bc10b-77c4-42e4-9c01-ed1fbe877339 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:23.477597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bc10b-77c4-42e4-9c01-ed1fbe877339 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:23.478266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bc10b-77c4-42e4-9c01-ed1fbe877339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:23.493864 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bc10b-77c4-42e4-9c01-ed1fbe877339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:23.495955 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bc10b-77c4-42e4-9c01-ed1fbe877339 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:23.508409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 667/1327] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:23.847142 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c2bb3-9723-4c6b-a384-7ed6fa08f59a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:23.850063 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6d6c2bb3-9723-4c6b-a384-7ed6fa08f59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:23.885120 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c2bb3-9723-4c6b-a384-7ed6fa08f59a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:23.898705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b3b8e-e1a6-4342-a0ce-586d5616801c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:23.899718 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b3b8e-e1a6-4342-a0ce-586d5616801c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:23.900301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b3b8e-e1a6-4342-a0ce-586d5616801c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:23.901283 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b3b8e-e1a6-4342-a0ce-586d5616801c admin admin] PATH_INFO: `/v3/projects/b13c82f939b04b2bb2e370d5de09fe3f` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:23.914989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b3b8e-e1a6-4342-a0ce-586d5616801c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b13c82f939b04b2bb2e370d5de09fe3f)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:23.916074 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-672b3b8e-e1a6-4342-a0ce-586d5616801c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:24:23.916074 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 14 02:24:23.916074 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: Oct 14 02:24:23.923839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b3b8e-e1a6-4342-a0ce-586d5616801c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:24.095098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-672b3b8e-e1a6-4342-a0ce-586d5616801c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b13c82f939b04b2bb2e370d5de09fe3f', 'request_id': 'req-672b3b8e-e1a6-4342-a0ce-586d5616801c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:24.110704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-672b3b8e-e1a6-4342-a0ce-586d5616801c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b13c82f939b04b2bb2e370d5de09fe3f', 'request_id': 'req-672b3b8e-e1a6-4342-a0ce-586d5616801c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:24.169788 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-672b3b8e-e1a6-4342-a0ce-586d5616801c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b13c82f939b04b2bb2e370d5de09fe3f', 'request_id': 'req-672b3b8e-e1a6-4342-a0ce-586d5616801c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:24.196757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-672b3b8e-e1a6-4342-a0ce-586d5616801c admin admin] The token cache is being invalidate because project b13c82f939b04b2bb2e370d5de09fe3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:24.197321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-672b3b8e-e1a6-4342-a0ce-586d5616801c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:24.197937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-672b3b8e-e1a6-4342-a0ce-586d5616801c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:24.199913 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 661/1328] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:23 2022] DELETE /identity/v3/projects/b13c82f939b04b2bb2e370d5de09fe3f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:24.204453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ab07a-4809-40d5-a1fc-6623a1f4e49b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:24.205211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ab07a-4809-40d5-a1fc-6623a1f4e49b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:24.205645 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ab07a-4809-40d5-a1fc-6623a1f4e49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:24.248792 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0e8ab07a-4809-40d5-a1fc-6623a1f4e49b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:24.253669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0e8ab07a-4809-40d5-a1fc-6623a1f4e49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:24.412633 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 668/1329] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:24.414833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6ead9-ac7e-452f-b069-fea889fb0edf None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:24.415161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6ead9-ac7e-452f-b069-fea889fb0edf None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:24.415497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6ead9-ac7e-452f-b069-fea889fb0edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:24.461030 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-97a6ead9-ac7e-452f-b069-fea889fb0edf None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:24.466547 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-97a6ead9-ac7e-452f-b069-fea889fb0edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:24.487937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 662/1330] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:24.492843 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ab07a-4809-40d5-a1fc-6623a1f4e49b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:24.509508 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ab07a-4809-40d5-a1fc-6623a1f4e49b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:24.528517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d86fe-155b-4f2d-90d5-82c17b54923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:24.530663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d86fe-155b-4f2d-90d5-82c17b54923d admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:24.530964 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d86fe-155b-4f2d-90d5-82c17b54923d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:24.531282 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d86fe-155b-4f2d-90d5-82c17b54923d admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:24.531955 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1d86fe-155b-4f2d-90d5-82c17b54923d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:24.532262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1d86fe-155b-4f2d-90d5-82c17b54923d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:24.536391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1d86fe-155b-4f2d-90d5-82c17b54923d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:24.569793 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 669/1331] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:24.572218 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6ead9-ac7e-452f-b069-fea889fb0edf None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:24.584376 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6ead9-ac7e-452f-b069-fea889fb0edf None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:24.600780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f368c-0d1a-4a74-ade3-397f571057de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:24.601683 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f368c-0d1a-4a74-ade3-397f571057de admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:24.602322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f368c-0d1a-4a74-ade3-397f571057de admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:24.603259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f368c-0d1a-4a74-ade3-397f571057de admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:24.604047 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f368c-0d1a-4a74-ade3-397f571057de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:24.609187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f368c-0d1a-4a74-ade3-397f571057de admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:24.727535 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 663/1332] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:24 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:24.751206 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d86fe-155b-4f2d-90d5-82c17b54923d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:24.751206 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d86fe-155b-4f2d-90d5-82c17b54923d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:24.901457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6115621-a867-4954-a4dc-f3e71144260c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:24.902589 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6115621-a867-4954-a4dc-f3e71144260c admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:24.903735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6115621-a867-4954-a4dc-f3e71144260c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:24.904618 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6115621-a867-4954-a4dc-f3e71144260c admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:24.906271 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6115621-a867-4954-a4dc-f3e71144260c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:24.912817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6115621-a867-4954-a4dc-f3e71144260c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:24.963946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 670/1333] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:24 2022] POST /identity/v3/users => generated 283 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:24.967932 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f368c-0d1a-4a74-ade3-397f571057de admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:24.982062 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f368c-0d1a-4a74-ade3-397f571057de admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:24.995656 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7f688-88d7-4160-bafe-286f5e282ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:24.998424 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7f688-88d7-4160-bafe-286f5e282ebe admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:24.998707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7f688-88d7-4160-bafe-286f5e282ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:24.998991 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7f688-88d7-4160-bafe-286f5e282ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:25.000772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7f688-88d7-4160-bafe-286f5e282ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:25.008269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7f688-88d7-4160-bafe-286f5e282ebe admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:25.049791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 664/1334] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:25.055704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6115621-a867-4954-a4dc-f3e71144260c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:25.073694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6115621-a867-4954-a4dc-f3e71144260c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:25.089182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2cffa-e24a-487e-a4fc-bd888b28a5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:25.094161 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2cffa-e24a-487e-a4fc-bd888b28a5dc admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:25.097538 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2cffa-e24a-487e-a4fc-bd888b28a5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:25.099294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2cffa-e24a-487e-a4fc-bd888b28a5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:25.101759 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2cffa-e24a-487e-a4fc-bd888b28a5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:25.104656 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2cffa-e24a-487e-a4fc-bd888b28a5dc admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:25.132426 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 671/1335] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:24:25.134201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7f688-88d7-4160-bafe-286f5e282ebe admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:25.155968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7f688-88d7-4160-bafe-286f5e282ebe admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:25.166863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b3c2e-f026-4681-b2c4-c2c01cb2c437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:25.169013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b3c2e-f026-4681-b2c4-c2c01cb2c437 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:25.169546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b3c2e-f026-4681-b2c4-c2c01cb2c437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:25.170651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b3c2e-f026-4681-b2c4-c2c01cb2c437 admin admin] PATH_INFO: `/v3/projects/a8c9bec21f2c47b290ab18a4156152ff/users/9be053db30cd49eeba69b4d3a923cff0/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:25.198162 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b3c2e-f026-4681-b2c4-c2c01cb2c437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c9bec21f2c47b290ab18a4156152ff, user_id=9be053db30cd49eeba69b4d3a923cff0, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:25.205961 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b3c2e-f026-4681-b2c4-c2c01cb2c437 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:25.257409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f03a43-1971-48bb-ad25-17fb436606a9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:25.257926 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f03a43-1971-48bb-ad25-17fb436606a9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:25.258152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f03a43-1971-48bb-ad25-17fb436606a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:25.267635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 665/1336] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:25 2022] PUT /identity/v3/projects/a8c9bec21f2c47b290ab18a4156152ff/users/9be053db30cd49eeba69b4d3a923cff0/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:25.290574 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:25.341893 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b3c2e-f026-4681-b2c4-c2c01cb2c437 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:25.344712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-de3b3c2e-f026-4681-b2c4-c2c01cb2c437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:25.362796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-74f03a43-1971-48bb-ad25-17fb436606a9 None None] MFA Rules not processed for user `9be053db30cd49eeba69b4d3a923cff0`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:25.364340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-74f03a43-1971-48bb-ad25-17fb436606a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:25.556896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b3c2e-f026-4681-b2c4-c2c01cb2c437 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:25.569479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40293543-0283-465c-b128-05d196ec1c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:25.570173 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40293543-0283-465c-b128-05d196ec1c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:25.570448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40293543-0283-465c-b128-05d196ec1c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:25.570733 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40293543-0283-465c-b128-05d196ec1c62 admin admin] PATH_INFO: `/v3/users/fbea6f3499804746b0daa91860584f43` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:25.572159 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40293543-0283-465c-b128-05d196ec1c62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbea6f3499804746b0daa91860584f43)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:25.576523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40293543-0283-465c-b128-05d196ec1c62 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:25.624837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 672/1337] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:25.634936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f03a43-1971-48bb-ad25-17fb436606a9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:25.638031 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-74f03a43-1971-48bb-ad25-17fb436606a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:25.647436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:25.708733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f03a43-1971-48bb-ad25-17fb436606a9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:25.719101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7c87e-3454-4de9-a955-8cbf8d2317cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:25.728063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7c87e-3454-4de9-a955-8cbf8d2317cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:25.728958 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7c87e-3454-4de9-a955-8cbf8d2317cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:25.729710 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7c87e-3454-4de9-a955-8cbf8d2317cf admin admin] PATH_INFO: `/v3/projects/31425e92610a4541b2e74b733f6b32e8` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:25.732553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7c87e-3454-4de9-a955-8cbf8d2317cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=31425e92610a4541b2e74b733f6b32e8)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:25.736882 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7c87e-3454-4de9-a955-8cbf8d2317cf admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:25.765521 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-40293543-0283-465c-b128-05d196ec1c62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbea6f3499804746b0daa91860584f43'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:25.786570 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-40293543-0283-465c-b128-05d196ec1c62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbea6f3499804746b0daa91860584f43'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:25.801453 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-40293543-0283-465c-b128-05d196ec1c62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbea6f3499804746b0daa91860584f43'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:25.840896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-40293543-0283-465c-b128-05d196ec1c62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbea6f3499804746b0daa91860584f43'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:25.855461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-50b7c87e-3454-4de9-a955-8cbf8d2317cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31425e92610a4541b2e74b733f6b32e8', 'request_id': 'req-50b7c87e-3454-4de9-a955-8cbf8d2317cf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:25.866444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-40293543-0283-465c-b128-05d196ec1c62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbea6f3499804746b0daa91860584f43'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:25.890355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-50b7c87e-3454-4de9-a955-8cbf8d2317cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31425e92610a4541b2e74b733f6b32e8', 'request_id': 'req-50b7c87e-3454-4de9-a955-8cbf8d2317cf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:25.905812 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 666/1338] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:25 2022] DELETE /identity/v3/users/fbea6f3499804746b0daa91860584f43 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:25.907856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40293543-0283-465c-b128-05d196ec1c62 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:25.910534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-40293543-0283-465c-b128-05d196ec1c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:25.914008 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-50b7c87e-3454-4de9-a955-8cbf8d2317cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31425e92610a4541b2e74b733f6b32e8', 'request_id': 'req-50b7c87e-3454-4de9-a955-8cbf8d2317cf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:25.957917 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-50b7c87e-3454-4de9-a955-8cbf8d2317cf admin admin] The token cache is being invalidate because project 31425e92610a4541b2e74b733f6b32e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:25.958781 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-50b7c87e-3454-4de9-a955-8cbf8d2317cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:25.966166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-50b7c87e-3454-4de9-a955-8cbf8d2317cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:25.971906 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 673/1339] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:25 2022] DELETE /identity/v3/projects/31425e92610a4541b2e74b733f6b32e8 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:25.980118 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51333410-bc70-475b-8d93-5f081b07c563 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:25.980368 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51333410-bc70-475b-8d93-5f081b07c563 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:25.980795 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51333410-bc70-475b-8d93-5f081b07c563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:26.058580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-51333410-bc70-475b-8d93-5f081b07c563 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:26.063730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-51333410-bc70-475b-8d93-5f081b07c563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:26.120747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40293543-0283-465c-b128-05d196ec1c62 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:26.143783 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680a8d1-1841-4263-a08b-8386b76194a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:26.144430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680a8d1-1841-4263-a08b-8386b76194a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:26.150647 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680a8d1-1841-4263-a08b-8386b76194a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:26.152361 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680a8d1-1841-4263-a08b-8386b76194a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:26.153088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680a8d1-1841-4263-a08b-8386b76194a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:26.155120 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680a8d1-1841-4263-a08b-8386b76194a5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:26.210245 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 667/1340] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:25 2022] POST /identity/v3/projects => generated 365 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:26.211909 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680a8d1-1841-4263-a08b-8386b76194a5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:26.214886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d680a8d1-1841-4263-a08b-8386b76194a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:26.251414 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 674/1341] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:26.252779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51333410-bc70-475b-8d93-5f081b07c563 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:26.254380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-51333410-bc70-475b-8d93-5f081b07c563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:26.268638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51333410-bc70-475b-8d93-5f081b07c563 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:26.295668 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:26.377816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680a8d1-1841-4263-a08b-8386b76194a5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:26.390351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42399a-e2fe-4e45-93c7-67a736d51946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:26.391883 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42399a-e2fe-4e45-93c7-67a736d51946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:26.392784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42399a-e2fe-4e45-93c7-67a736d51946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:26.393693 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42399a-e2fe-4e45-93c7-67a736d51946 admin admin] PATH_INFO: `/v3/users/7cebd05c17824d478e57138e521a5e11` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:26.397637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42399a-e2fe-4e45-93c7-67a736d51946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cebd05c17824d478e57138e521a5e11)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:26.401453 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42399a-e2fe-4e45-93c7-67a736d51946 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:26.402343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a0896-0073-4c8c-8c4b-1a503a98fb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:26.403181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a0896-0073-4c8c-8c4b-1a503a98fb7e admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:26.404870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a0896-0073-4c8c-8c4b-1a503a98fb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:26.405301 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a0896-0073-4c8c-8c4b-1a503a98fb7e admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:26.406071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a0896-0073-4c8c-8c4b-1a503a98fb7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:26.411444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a0896-0073-4c8c-8c4b-1a503a98fb7e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:26.468744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 675/1342] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:26 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:26.470330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:26.473502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a0896-0073-4c8c-8c4b-1a503a98fb7e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:26.485123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-967a0896-0073-4c8c-8c4b-1a503a98fb7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:26.553360 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd42399a-e2fe-4e45-93c7-67a736d51946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cebd05c17824d478e57138e521a5e11'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:26.571049 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd42399a-e2fe-4e45-93c7-67a736d51946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cebd05c17824d478e57138e521a5e11'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:26.572009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd42399a-e2fe-4e45-93c7-67a736d51946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cebd05c17824d478e57138e521a5e11'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:26.598844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd42399a-e2fe-4e45-93c7-67a736d51946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cebd05c17824d478e57138e521a5e11'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:26.615082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd42399a-e2fe-4e45-93c7-67a736d51946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cebd05c17824d478e57138e521a5e11'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:26.625929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 668/1343] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:26 2022] DELETE /identity/v3/users/7cebd05c17824d478e57138e521a5e11 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:26.627281 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42399a-e2fe-4e45-93c7-67a736d51946 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:26.629477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cd42399a-e2fe-4e45-93c7-67a736d51946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:26.671914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a0896-0073-4c8c-8c4b-1a503a98fb7e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:26.821634 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42399a-e2fe-4e45-93c7-67a736d51946 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:26.838977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2828f-6e41-4e14-b3b3-8b9d17762f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:26.841836 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2828f-6e41-4e14-b3b3-8b9d17762f94 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:26.842209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2828f-6e41-4e14-b3b3-8b9d17762f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:26.842560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2828f-6e41-4e14-b3b3-8b9d17762f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:26.844209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2828f-6e41-4e14-b3b3-8b9d17762f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:26.849675 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2828f-6e41-4e14-b3b3-8b9d17762f94 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:26.877724 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 669/1344] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:26.881002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2828f-6e41-4e14-b3b3-8b9d17762f94 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:26.885626 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5bb01-88cb-4a5f-a5a4-292350fcfcd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:26.886267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5bb01-88cb-4a5f-a5a4-292350fcfcd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:26.886544 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5bb01-88cb-4a5f-a5a4-292350fcfcd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:26.886815 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5bb01-88cb-4a5f-a5a4-292350fcfcd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:26.891469 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-33e5bb01-88cb-4a5f-a5a4-292350fcfcd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:26.906121 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2828f-6e41-4e14-b3b3-8b9d17762f94 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:26.918512 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988b68e-b64c-4647-8729-84af4025d2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:26.920233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988b68e-b64c-4647-8729-84af4025d2ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:26.921200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988b68e-b64c-4647-8729-84af4025d2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:26.921658 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988b68e-b64c-4647-8729-84af4025d2ea admin admin] PATH_INFO: `/v3/projects/1f408bbfbd60422e863605b5fbe1878f/users/05777325449745f89eca267f9006c2a2/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:26.939027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5bb01-88cb-4a5f-a5a4-292350fcfcd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:26.940551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5bb01-88cb-4a5f-a5a4-292350fcfcd6 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:26.977554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988b68e-b64c-4647-8729-84af4025d2ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f408bbfbd60422e863605b5fbe1878f, user_id=05777325449745f89eca267f9006c2a2, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:26.981674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988b68e-b64c-4647-8729-84af4025d2ea admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:26.984572 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 676/1345] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:27.025726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 670/1346] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:26 2022] PUT /identity/v3/projects/1f408bbfbd60422e863605b5fbe1878f/users/05777325449745f89eca267f9006c2a2/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:27.026996 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5bb01-88cb-4a5f-a5a4-292350fcfcd6 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:27.039668 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5bb01-88cb-4a5f-a5a4-292350fcfcd6 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:27.217867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6ebd5-dd29-40a9-a6e4-689a0e5ed37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:27.218809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6ebd5-dd29-40a9-a6e4-689a0e5ed37a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:27.219345 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6ebd5-dd29-40a9-a6e4-689a0e5ed37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:27.219783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6ebd5-dd29-40a9-a6e4-689a0e5ed37a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:27.220761 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f6ebd5-dd29-40a9-a6e4-689a0e5ed37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:27.222653 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f6ebd5-dd29-40a9-a6e4-689a0e5ed37a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:27.261006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 677/1347] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:27.265091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988b68e-b64c-4647-8729-84af4025d2ea admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:27.281321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988b68e-b64c-4647-8729-84af4025d2ea admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:27.294996 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240c61d-22ca-4884-9028-4c2b59f0a7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:27.295665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240c61d-22ca-4884-9028-4c2b59f0a7d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:27.295943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240c61d-22ca-4884-9028-4c2b59f0a7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:27.296215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240c61d-22ca-4884-9028-4c2b59f0a7d1 admin admin] PATH_INFO: `/v3/projects/1f408bbfbd60422e863605b5fbe1878f/users/05777325449745f89eca267f9006c2a2/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:27.300995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240c61d-22ca-4884-9028-4c2b59f0a7d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f408bbfbd60422e863605b5fbe1878f, user_id=05777325449745f89eca267f9006c2a2, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:27.304286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240c61d-22ca-4884-9028-4c2b59f0a7d1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:27.346633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 671/1348] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:27 2022] PUT /identity/v3/projects/1f408bbfbd60422e863605b5fbe1878f/users/05777325449745f89eca267f9006c2a2/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:24:27.357052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b7445-63d3-4d04-ad7d-d92ad6642da2 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:27.358043 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b7445-63d3-4d04-ad7d-d92ad6642da2 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:27.358484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b7445-63d3-4d04-ad7d-d92ad6642da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:27.369728 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:27.459192 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-be7b7445-63d3-4d04-ad7d-d92ad6642da2 None None] MFA Rules not processed for user `05777325449745f89eca267f9006c2a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:27.461152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-be7b7445-63d3-4d04-ad7d-d92ad6642da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:27.685418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 678/1349] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:27 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:27.693102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240c61d-22ca-4884-9028-4c2b59f0a7d1 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:27.694902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d240c61d-22ca-4884-9028-4c2b59f0a7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:27.817440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240c61d-22ca-4884-9028-4c2b59f0a7d1 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:27.823697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdbc67-39ec-4277-9ecd-00f0f885aa9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:27.824422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdbc67-39ec-4277-9ecd-00f0f885aa9a service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:27.824765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdbc67-39ec-4277-9ecd-00f0f885aa9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:27.825293 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdbc67-39ec-4277-9ecd-00f0f885aa9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:27.863144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdbc67-39ec-4277-9ecd-00f0f885aa9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:27.866916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdbc67-39ec-4277-9ecd-00f0f885aa9a service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:27.876392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:27.896275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 672/1350] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:28.021516 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b7445-63d3-4d04-ad7d-d92ad6642da2 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:28.054549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b7445-63d3-4d04-ad7d-d92ad6642da2 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:28.075087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a6d55-48d5-49ca-87da-0f1a223cbd6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:28.076156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a6d55-48d5-49ca-87da-0f1a223cbd6a service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:28.077626 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a6d55-48d5-49ca-87da-0f1a223cbd6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:28.078100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a6d55-48d5-49ca-87da-0f1a223cbd6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:28.082317 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e18a6d55-48d5-49ca-87da-0f1a223cbd6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:28.273106 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a6d55-48d5-49ca-87da-0f1a223cbd6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:28.275282 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a6d55-48d5-49ca-87da-0f1a223cbd6a service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:28.305537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 679/1351] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:29.024768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdbc67-39ec-4277-9ecd-00f0f885aa9a service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:29.027144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dcbdbc67-39ec-4277-9ecd-00f0f885aa9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:29.119899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdbc67-39ec-4277-9ecd-00f0f885aa9a service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:29.125432 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:29.126749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:29.127268 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:29.127554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07 admin admin] PATH_INFO: `/v3/projects/dcbfe035b11a424a827cb5522e6ef8f2` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:29.128990 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcbfe035b11a424a827cb5522e6ef8f2)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:29.132404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:29.222728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcbfe035b11a424a827cb5522e6ef8f2', 'request_id': 'req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:29.239126 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcbfe035b11a424a827cb5522e6ef8f2', 'request_id': 'req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:29.273434 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcbfe035b11a424a827cb5522e6ef8f2', 'request_id': 'req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:29.296489 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07 admin admin] The token cache is being invalidate because project dcbfe035b11a424a827cb5522e6ef8f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:29.297368 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:29.298353 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2e2e3e9d-ceb6-4fdb-a173-dfabb3e4bc07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:29.300645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 673/1352] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:29 2022] DELETE /identity/v3/projects/dcbfe035b11a424a827cb5522e6ef8f2 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:29.305875 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dd9d9-ea12-4795-9a92-975f105ac39e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:29.306349 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dd9d9-ea12-4795-9a92-975f105ac39e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:29.306442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dd9d9-ea12-4795-9a92-975f105ac39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:29.340402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bd3dd9d9-ea12-4795-9a92-975f105ac39e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:29.341777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bd3dd9d9-ea12-4795-9a92-975f105ac39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:29.497862 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 680/1353] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:29.506881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b29e3a-1932-405d-89cb-f5b2b819b18c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:29.507645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b29e3a-1932-405d-89cb-f5b2b819b18c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:29.509960 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b29e3a-1932-405d-89cb-f5b2b819b18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:29.550425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-47b29e3a-1932-405d-89cb-f5b2b819b18c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:29.554437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-47b29e3a-1932-405d-89cb-f5b2b819b18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:29.560033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3dd9d9-ea12-4795-9a92-975f105ac39e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:29.561866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bd3dd9d9-ea12-4795-9a92-975f105ac39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:29.587653 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3dd9d9-ea12-4795-9a92-975f105ac39e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:29.594894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 674/1354] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:24:29.599567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b29e3a-1932-405d-89cb-f5b2b819b18c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:29.600538 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8e77b-4c50-46ad-aa4c-29593bb64346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:29.601213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8e77b-4c50-46ad-aa4c-29593bb64346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:29.601491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8e77b-4c50-46ad-aa4c-29593bb64346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:29.601764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8e77b-4c50-46ad-aa4c-29593bb64346 admin admin] PATH_INFO: `/v3/projects/e81bc6b4efe24f1da9b1d75c335bb73e` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:29.604048 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8e77b-4c50-46ad-aa4c-29593bb64346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e81bc6b4efe24f1da9b1d75c335bb73e)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:29.606313 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8e77b-4c50-46ad-aa4c-29593bb64346 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:29.618009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b29e3a-1932-405d-89cb-f5b2b819b18c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:29.635684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba0b9d-ef05-4cb5-b2e4-a9dac66001e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:29.636322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba0b9d-ef05-4cb5-b2e4-a9dac66001e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:29.636601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba0b9d-ef05-4cb5-b2e4-a9dac66001e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:29.636871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba0b9d-ef05-4cb5-b2e4-a9dac66001e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:29.637522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba0b9d-ef05-4cb5-b2e4-a9dac66001e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:29.637822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba0b9d-ef05-4cb5-b2e4-a9dac66001e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:29.639538 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba0b9d-ef05-4cb5-b2e4-a9dac66001e6 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:29.654513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:29.665217 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 675/1355] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:24:29.669238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba0b9d-ef05-4cb5-b2e4-a9dac66001e6 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:29.694837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba0b9d-ef05-4cb5-b2e4-a9dac66001e6 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:29.705211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c3a8e77b-4c50-46ad-aa4c-29593bb64346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e81bc6b4efe24f1da9b1d75c335bb73e', 'request_id': 'req-c3a8e77b-4c50-46ad-aa4c-29593bb64346'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:29.729224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c3a8e77b-4c50-46ad-aa4c-29593bb64346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e81bc6b4efe24f1da9b1d75c335bb73e', 'request_id': 'req-c3a8e77b-4c50-46ad-aa4c-29593bb64346'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:29.760065 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c3a8e77b-4c50-46ad-aa4c-29593bb64346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e81bc6b4efe24f1da9b1d75c335bb73e', 'request_id': 'req-c3a8e77b-4c50-46ad-aa4c-29593bb64346'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:29.794401 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c3a8e77b-4c50-46ad-aa4c-29593bb64346 admin admin] The token cache is being invalidate because project e81bc6b4efe24f1da9b1d75c335bb73e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:29.794866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c3a8e77b-4c50-46ad-aa4c-29593bb64346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:29.797817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c3a8e77b-4c50-46ad-aa4c-29593bb64346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:29.807259 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 681/1356] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:29 2022] DELETE /identity/v3/projects/e81bc6b4efe24f1da9b1d75c335bb73e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:29.808720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1c484-08d6-435b-84cc-55dfebe60b1b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:29.809013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1c484-08d6-435b-84cc-55dfebe60b1b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:29.809294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1c484-08d6-435b-84cc-55dfebe60b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:29.837192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a4747-e863-423e-851a-862aa06112bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:29.837995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a4747-e863-423e-851a-862aa06112bf admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:29.838202 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a4747-e863-423e-851a-862aa06112bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:29.838480 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a4747-e863-423e-851a-862aa06112bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:29.839183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a4747-e863-423e-851a-862aa06112bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:29.841109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a4747-e863-423e-851a-862aa06112bf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:29.849836 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5db1c484-08d6-435b-84cc-55dfebe60b1b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:29.855283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5db1c484-08d6-435b-84cc-55dfebe60b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:29.877863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 682/1357] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:29.879347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afdcb4-1352-4cc4-82a4-c78c35c18df9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:29.880218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afdcb4-1352-4cc4-82a4-c78c35c18df9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:29.880671 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afdcb4-1352-4cc4-82a4-c78c35c18df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:29.884812 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:29.911676 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 676/1358] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:29 2022] POST /identity/v3/projects => generated 365 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:24:29.912644 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a4747-e863-423e-851a-862aa06112bf admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:29.914512 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d7afdcb4-1352-4cc4-82a4-c78c35c18df9 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:29.915999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d7afdcb4-1352-4cc4-82a4-c78c35c18df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:29.919888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6f2a4747-e863-423e-851a-862aa06112bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:30.025101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 683/1359] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:30.030597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afdcb4-1352-4cc4-82a4-c78c35c18df9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:30.043871 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afdcb4-1352-4cc4-82a4-c78c35c18df9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:30.052285 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7682582-cb44-4233-ad33-e49c89a570bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:30.053556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7682582-cb44-4233-ad33-e49c89a570bc admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:30.054070 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7682582-cb44-4233-ad33-e49c89a570bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:30.054552 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7682582-cb44-4233-ad33-e49c89a570bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:30.055517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7682582-cb44-4233-ad33-e49c89a570bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:30.056037 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7682582-cb44-4233-ad33-e49c89a570bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:30.058238 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7682582-cb44-4233-ad33-e49c89a570bc admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:30.084432 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a4747-e863-423e-851a-862aa06112bf admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:30.092418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 684/1360] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:30.094307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7682582-cb44-4233-ad33-e49c89a570bc admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:30.097495 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a813c-e368-4bfe-8cec-f4b3f85d249f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:30.098462 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a813c-e368-4bfe-8cec-f4b3f85d249f admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:30.098914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a813c-e368-4bfe-8cec-f4b3f85d249f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:30.099362 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a813c-e368-4bfe-8cec-f4b3f85d249f admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:30.100330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a813c-e368-4bfe-8cec-f4b3f85d249f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:30.102464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a813c-e368-4bfe-8cec-f4b3f85d249f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:30.124402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7682582-cb44-4233-ad33-e49c89a570bc admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:30.140420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf461fe-cf92-4bd6-a3a4-ea8c8d1b58ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:30.144723 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf461fe-cf92-4bd6-a3a4-ea8c8d1b58ad admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:30.145287 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf461fe-cf92-4bd6-a3a4-ea8c8d1b58ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:30.146079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf461fe-cf92-4bd6-a3a4-ea8c8d1b58ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:30.146960 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf461fe-cf92-4bd6-a3a4-ea8c8d1b58ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:30.149359 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf461fe-cf92-4bd6-a3a4-ea8c8d1b58ad admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:30.149722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 677/1361] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:29 2022] POST /identity/v3/users => generated 283 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:30.154172 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a813c-e368-4bfe-8cec-f4b3f85d249f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:30.173595 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a813c-e368-4bfe-8cec-f4b3f85d249f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:30.185907 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b276d4-45ef-406f-9a04-e2e44df27e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:30.186553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b276d4-45ef-406f-9a04-e2e44df27e2d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:30.186835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b276d4-45ef-406f-9a04-e2e44df27e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:30.187129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b276d4-45ef-406f-9a04-e2e44df27e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:30.187971 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b276d4-45ef-406f-9a04-e2e44df27e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:30.195704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b276d4-45ef-406f-9a04-e2e44df27e2d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:30.223703 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 685/1362] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:30 2022] POST /identity/v3/projects => generated 403 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:30.225116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf461fe-cf92-4bd6-a3a4-ea8c8d1b58ad admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:30.232332 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 678/1363] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:24:30.238821 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b276d4-45ef-406f-9a04-e2e44df27e2d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:30.265253 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b276d4-45ef-406f-9a04-e2e44df27e2d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:30.408241 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf461fe-cf92-4bd6-a3a4-ea8c8d1b58ad admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:30.423235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54e698-4619-4b52-ab22-64f7b4d506f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:30.424203 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54e698-4619-4b52-ab22-64f7b4d506f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:30.424488 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54e698-4619-4b52-ab22-64f7b4d506f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:30.424768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54e698-4619-4b52-ab22-64f7b4d506f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:30.425420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54e698-4619-4b52-ab22-64f7b4d506f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:30.432765 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54e698-4619-4b52-ab22-64f7b4d506f8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:30.436577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10916f24-7e04-478c-ac11-47345ad0950e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:30.443859 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10916f24-7e04-478c-ac11-47345ad0950e admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:30.444384 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10916f24-7e04-478c-ac11-47345ad0950e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:30.444836 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10916f24-7e04-478c-ac11-47345ad0950e admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:30.445727 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10916f24-7e04-478c-ac11-47345ad0950e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:30.447770 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10916f24-7e04-478c-ac11-47345ad0950e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:30.476931 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 679/1364] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:30.487838 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10916f24-7e04-478c-ac11-47345ad0950e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:30.488589 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:30.508011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10916f24-7e04-478c-ac11-47345ad0950e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:30.514561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 686/1365] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:30 2022] POST /identity/v3/users => generated 302 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:30.520912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc2726-63b4-4c31-8715-df30db6ab7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:30.521608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54e698-4619-4b52-ab22-64f7b4d506f8 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:30.522144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc2726-63b4-4c31-8715-df30db6ab7b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:30.522825 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc2726-63b4-4c31-8715-df30db6ab7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:30.524073 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc2726-63b4-4c31-8715-df30db6ab7b8 admin admin] PATH_INFO: `/v3/projects/ed9569e193e74dbba29d3b0db1ffe0a1/users/fdb61cf00ee2481193bebea2e3e20c25/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:30.554205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54e698-4619-4b52-ab22-64f7b4d506f8 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:30.555226 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc2726-63b4-4c31-8715-df30db6ab7b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed9569e193e74dbba29d3b0db1ffe0a1, user_id=fdb61cf00ee2481193bebea2e3e20c25, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:30.559879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc2726-63b4-4c31-8715-df30db6ab7b8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:30.565553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63928b-ba13-44e4-96c9-0cd5462a9caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:30.568161 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63928b-ba13-44e4-96c9-0cd5462a9caf admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:30.568459 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63928b-ba13-44e4-96c9-0cd5462a9caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:30.570316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63928b-ba13-44e4-96c9-0cd5462a9caf admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:30.573452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63928b-ba13-44e4-96c9-0cd5462a9caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:30.577458 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63928b-ba13-44e4-96c9-0cd5462a9caf admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:30.584459 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 680/1366] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:30 2022] PUT /identity/v3/projects/ed9569e193e74dbba29d3b0db1ffe0a1/users/fdb61cf00ee2481193bebea2e3e20c25/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:30.596177 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c870586-62ed-4428-9938-2e0cda4e886b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:30.597800 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c870586-62ed-4428-9938-2e0cda4e886b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:30.600008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c870586-62ed-4428-9938-2e0cda4e886b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:30.622340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 687/1367] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:30.624432 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63928b-ba13-44e4-96c9-0cd5462a9caf admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:30.649788 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63928b-ba13-44e4-96c9-0cd5462a9caf admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:30.727034 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-2c870586-62ed-4428-9938-2e0cda4e886b None None] MFA Rules not processed for user `fdb61cf00ee2481193bebea2e3e20c25`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:30.737475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2c870586-62ed-4428-9938-2e0cda4e886b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:30.800265 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cec820-abac-4f8c-8e7e-9129bd720a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:30.801073 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cec820-abac-4f8c-8e7e-9129bd720a6a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:30.801548 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cec820-abac-4f8c-8e7e-9129bd720a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:30.802153 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cec820-abac-4f8c-8e7e-9129bd720a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:30.802966 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cec820-abac-4f8c-8e7e-9129bd720a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:30.805190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cec820-abac-4f8c-8e7e-9129bd720a6a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:30.827762 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 688/1368] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:30.831891 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cec820-abac-4f8c-8e7e-9129bd720a6a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:30.861644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cec820-abac-4f8c-8e7e-9129bd720a6a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:30.876014 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e09f7d-af91-476b-a99a-636ecc88b948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:30.877287 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e09f7d-af91-476b-a99a-636ecc88b948 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:30.877729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e09f7d-af91-476b-a99a-636ecc88b948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:30.878150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e09f7d-af91-476b-a99a-636ecc88b948 admin admin] PATH_INFO: `/v3/projects/e6596d5252d94376877fbccbe8918ca6/users/4587d8f0deae42f6a64ba824bc720e0e/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:30.893985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:30.901408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e09f7d-af91-476b-a99a-636ecc88b948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6596d5252d94376877fbccbe8918ca6, user_id=4587d8f0deae42f6a64ba824bc720e0e, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:30.904669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e09f7d-af91-476b-a99a-636ecc88b948 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:30.949052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 689/1369] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:30 2022] PUT /identity/v3/projects/e6596d5252d94376877fbccbe8918ca6/users/4587d8f0deae42f6a64ba824bc720e0e/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:30.965766 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19795761-37ea-4af2-ace0-d8ebc4070305 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:30.967779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19795761-37ea-4af2-ace0-d8ebc4070305 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:30.968256 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19795761-37ea-4af2-ace0-d8ebc4070305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:31.021791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 681/1370] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:31.023586 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c870586-62ed-4428-9938-2e0cda4e886b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:31.057397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-19795761-37ea-4af2-ace0-d8ebc4070305 None None] MFA Rules not processed for user `4587d8f0deae42f6a64ba824bc720e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:31.058527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c870586-62ed-4428-9938-2e0cda4e886b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:31.062454 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-19795761-37ea-4af2-ace0-d8ebc4070305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:31.214522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd59e80-cbbf-4acd-9445-85d25b0d84c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:31.215329 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd59e80-cbbf-4acd-9445-85d25b0d84c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:31.215638 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd59e80-cbbf-4acd-9445-85d25b0d84c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:31.216600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd59e80-cbbf-4acd-9445-85d25b0d84c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:31.217271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd59e80-cbbf-4acd-9445-85d25b0d84c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:31.222801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd59e80-cbbf-4acd-9445-85d25b0d84c3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:31.240270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 690/1371] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:30 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:31.244807 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19795761-37ea-4af2-ace0-d8ebc4070305 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:31.268069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19795761-37ea-4af2-ace0-d8ebc4070305 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:31.279667 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2cde7a-7500-4bf0-ba67-5df63ac9e40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:31.281938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cde7a-7500-4bf0-ba67-5df63ac9e40d admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:31.282334 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cde7a-7500-4bf0-ba67-5df63ac9e40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:31.282748 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cde7a-7500-4bf0-ba67-5df63ac9e40d admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:31.284136 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2cde7a-7500-4bf0-ba67-5df63ac9e40d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:31.290427 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2cde7a-7500-4bf0-ba67-5df63ac9e40d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:31.294023 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 682/1372] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:31 2022] POST /identity/v3/projects => generated 365 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:31.298499 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd59e80-cbbf-4acd-9445-85d25b0d84c3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:31.350050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 691/1373] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:31 2022] POST /identity/v3/projects => generated 403 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:24:31.352926 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2cde7a-7500-4bf0-ba67-5df63ac9e40d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:31.387240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd59e80-cbbf-4acd-9445-85d25b0d84c3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:31.429132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:31.494064 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92254715-ccf5-4c9e-a239-80d39162e8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:31.495010 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92254715-ccf5-4c9e-a239-80d39162e8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:31.495511 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92254715-ccf5-4c9e-a239-80d39162e8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:31.496910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92254715-ccf5-4c9e-a239-80d39162e8f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:31.498409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92254715-ccf5-4c9e-a239-80d39162e8f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:31.502290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92254715-ccf5-4c9e-a239-80d39162e8f2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:31.529011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:31.532451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2cde7a-7500-4bf0-ba67-5df63ac9e40d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:31.543268 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 683/1374] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:31 2022] POST /identity/v3/users => generated 289 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:31.548685 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ab684-6eec-4702-af64-b62e635672a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:31.549330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92254715-ccf5-4c9e-a239-80d39162e8f2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:31.549452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ab684-6eec-4702-af64-b62e635672a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:31.549657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ab684-6eec-4702-af64-b62e635672a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:31.549901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ab684-6eec-4702-af64-b62e635672a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:31.550543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ab684-6eec-4702-af64-b62e635672a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:31.556443 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ab684-6eec-4702-af64-b62e635672a1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:31.568082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92254715-ccf5-4c9e-a239-80d39162e8f2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:31.587413 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc34848-4cc6-4e76-bbdf-fdd43ebe0f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:31.589854 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc34848-4cc6-4e76-bbdf-fdd43ebe0f30 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:31.590594 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc34848-4cc6-4e76-bbdf-fdd43ebe0f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:31.592644 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc34848-4cc6-4e76-bbdf-fdd43ebe0f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:31.595151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc34848-4cc6-4e76-bbdf-fdd43ebe0f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:31.608255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc34848-4cc6-4e76-bbdf-fdd43ebe0f30 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:31.614385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 692/1375] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:31 2022] POST /identity/v3/users => generated 308 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:31.618332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ab684-6eec-4702-af64-b62e635672a1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:31.642865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ab684-6eec-4702-af64-b62e635672a1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:31.643625 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 684/1376] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:31.647207 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc34848-4cc6-4e76-bbdf-fdd43ebe0f30 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:31.664571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6b3a6-4f4b-4b03-84bb-8065b24b6cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:31.666217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6b3a6-4f4b-4b03-84bb-8065b24b6cef admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:31.666673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6b3a6-4f4b-4b03-84bb-8065b24b6cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:31.666948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6b3a6-4f4b-4b03-84bb-8065b24b6cef admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:31.667880 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6b3a6-4f4b-4b03-84bb-8065b24b6cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:31.674635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6b3a6-4f4b-4b03-84bb-8065b24b6cef admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:31.678298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc34848-4cc6-4e76-bbdf-fdd43ebe0f30 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:31.687740 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3fad3-eb58-4534-96d5-1196df3f82cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:31.689440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3fad3-eb58-4534-96d5-1196df3f82cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:31.690418 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3fad3-eb58-4534-96d5-1196df3f82cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:31.690975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3fad3-eb58-4534-96d5-1196df3f82cf admin admin] PATH_INFO: `/v3/projects/284bb33fe40444238f85b8d689ef46e2/users/f01cf12cf3e14a7b9eaeb9d4bb83289d/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:31.698044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 693/1377] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:24:31.702091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6b3a6-4f4b-4b03-84bb-8065b24b6cef admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:31.722655 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6b3a6-4f4b-4b03-84bb-8065b24b6cef admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:31.726750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3fad3-eb58-4534-96d5-1196df3f82cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=284bb33fe40444238f85b8d689ef46e2, user_id=f01cf12cf3e14a7b9eaeb9d4bb83289d, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:31.731427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3fad3-eb58-4534-96d5-1196df3f82cf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:31.733825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace5b95-d7f4-4c84-b9e4-fd2dce7a409c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:31.734513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace5b95-d7f4-4c84-b9e4-fd2dce7a409c admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:31.734790 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace5b95-d7f4-4c84-b9e4-fd2dce7a409c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:31.735996 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace5b95-d7f4-4c84-b9e4-fd2dce7a409c admin admin] PATH_INFO: `/v3/projects/587d9c00612f44498152cb13f5d1fe12/users/2577d4b0bfab4308b2869349c21fc1fa/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:31.779113 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 685/1378] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:31 2022] PUT /identity/v3/projects/284bb33fe40444238f85b8d689ef46e2/users/f01cf12cf3e14a7b9eaeb9d4bb83289d/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:31.783176 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3fad3-eb58-4534-96d5-1196df3f82cf admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:31.797050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace5b95-d7f4-4c84-b9e4-fd2dce7a409c admin admin] RBAC: Authorizing `identity:create_grant(project_id=587d9c00612f44498152cb13f5d1fe12, user_id=2577d4b0bfab4308b2869349c21fc1fa, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:31.813092 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace5b95-d7f4-4c84-b9e4-fd2dce7a409c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:31.854475 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 694/1379] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:31 2022] PUT /identity/v3/projects/587d9c00612f44498152cb13f5d1fe12/users/2577d4b0bfab4308b2869349c21fc1fa/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:31.858339 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace5b95-d7f4-4c84-b9e4-fd2dce7a409c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:32.015822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3fad3-eb58-4534-96d5-1196df3f82cf admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:32.079938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace5b95-d7f4-4c84-b9e4-fd2dce7a409c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:32.097923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71b50d-2331-4309-9cf1-b55911381cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:32.099042 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71b50d-2331-4309-9cf1-b55911381cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:32.105756 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71b50d-2331-4309-9cf1-b55911381cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:32.106111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71b50d-2331-4309-9cf1-b55911381cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:32.107043 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71b50d-2331-4309-9cf1-b55911381cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:32.109447 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71b50d-2331-4309-9cf1-b55911381cd5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:32.190391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 695/1380] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:32.197974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71b50d-2331-4309-9cf1-b55911381cd5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:32.247426 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71b50d-2331-4309-9cf1-b55911381cd5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:32.278438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533da89-631f-4b0c-b5da-1e72523d0474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:32.283569 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533da89-631f-4b0c-b5da-1e72523d0474 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:32.287712 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533da89-631f-4b0c-b5da-1e72523d0474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:32.288238 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533da89-631f-4b0c-b5da-1e72523d0474 admin admin] PATH_INFO: `/v3/projects/587d9c00612f44498152cb13f5d1fe12/users/2577d4b0bfab4308b2869349c21fc1fa/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:32.293710 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533da89-631f-4b0c-b5da-1e72523d0474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=587d9c00612f44498152cb13f5d1fe12, user_id=2577d4b0bfab4308b2869349c21fc1fa, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:32.304835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5ed16-94f5-4076-8173-b9311a260d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:32.305586 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5ed16-94f5-4076-8173-b9311a260d87 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:32.306070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5ed16-94f5-4076-8173-b9311a260d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:32.306689 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5ed16-94f5-4076-8173-b9311a260d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:32.307413 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5ed16-94f5-4076-8173-b9311a260d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:32.307998 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533da89-631f-4b0c-b5da-1e72523d0474 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:32.312392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5ed16-94f5-4076-8173-b9311a260d87 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:32.342503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 696/1381] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:32 2022] PUT /identity/v3/projects/587d9c00612f44498152cb13f5d1fe12/users/2577d4b0bfab4308b2869349c21fc1fa/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:32.345766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 686/1382] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 563 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:24:32.348531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533da89-631f-4b0c-b5da-1e72523d0474 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:32.356527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c63d2e-e03d-48a6-9a2d-f1d1769da7ff None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:32.356878 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c63d2e-e03d-48a6-9a2d-f1d1769da7ff None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:32.357145 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c63d2e-e03d-48a6-9a2d-f1d1769da7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:32.465698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c6c63d2e-e03d-48a6-9a2d-f1d1769da7ff None None] MFA Rules not processed for user `2577d4b0bfab4308b2869349c21fc1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:32.471467 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c6c63d2e-e03d-48a6-9a2d-f1d1769da7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:32.586002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533da89-631f-4b0c-b5da-1e72523d0474 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:32.601184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae121e-6c28-4dd9-8b48-f8eaa19c98b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:32.601890 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae121e-6c28-4dd9-8b48-f8eaa19c98b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:32.602171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae121e-6c28-4dd9-8b48-f8eaa19c98b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:32.602443 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae121e-6c28-4dd9-8b48-f8eaa19c98b9 admin admin] PATH_INFO: `/v3/projects/284bb33fe40444238f85b8d689ef46e2/users/f01cf12cf3e14a7b9eaeb9d4bb83289d/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:32.608016 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae121e-6c28-4dd9-8b48-f8eaa19c98b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=284bb33fe40444238f85b8d689ef46e2, user_id=f01cf12cf3e14a7b9eaeb9d4bb83289d, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:32.610644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae121e-6c28-4dd9-8b48-f8eaa19c98b9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:32.625103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:32.649093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 697/1383] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:32 2022] PUT /identity/v3/projects/284bb33fe40444238f85b8d689ef46e2/users/f01cf12cf3e14a7b9eaeb9d4bb83289d/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:32.650100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe56ca6-433d-4328-adf0-a815f157dd4f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:32.650404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe56ca6-433d-4328-adf0-a815f157dd4f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:32.650695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe56ca6-433d-4328-adf0-a815f157dd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:32.718345 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-dfe56ca6-433d-4328-adf0-a815f157dd4f None None] MFA Rules not processed for user `f01cf12cf3e14a7b9eaeb9d4bb83289d`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:32.720224 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:32.730290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-dfe56ca6-433d-4328-adf0-a815f157dd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:32.797907 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 687/1384] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:32 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 443 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:32.923453 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c63d2e-e03d-48a6-9a2d-f1d1769da7ff None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:32.925480 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c6c63d2e-e03d-48a6-9a2d-f1d1769da7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:33.001013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 698/1385] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:24:33.110956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c63d2e-e03d-48a6-9a2d-f1d1769da7ff None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:33.262870 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816aa625-dcfd-40a4-a9b8-bc702606e339 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:33.263825 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa625-dcfd-40a4-a9b8-bc702606e339 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:33.264444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa625-dcfd-40a4-a9b8-bc702606e339 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:33.264847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa625-dcfd-40a4-a9b8-bc702606e339 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:33.462004 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816aa625-dcfd-40a4-a9b8-bc702606e339 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:33.463571 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816aa625-dcfd-40a4-a9b8-bc702606e339 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:33.469262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe56ca6-433d-4328-adf0-a815f157dd4f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:33.471398 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-dfe56ca6-433d-4328-adf0-a815f157dd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:33.476789 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 688/1386] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:32 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:24:33.499942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe56ca6-433d-4328-adf0-a815f157dd4f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:33.507243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b387c-95bb-4775-91a9-f2fc4dc36c21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:33.508062 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b387c-95bb-4775-91a9-f2fc4dc36c21 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:33.519693 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b387c-95bb-4775-91a9-f2fc4dc36c21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:33.523848 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b387c-95bb-4775-91a9-f2fc4dc36c21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:33.561362 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b387c-95bb-4775-91a9-f2fc4dc36c21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:33.563223 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b387c-95bb-4775-91a9-f2fc4dc36c21 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:33.590875 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 699/1387] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:24:34.141417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2637d-fb6c-424b-b80d-bde2cdff4768 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:34.142583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2637d-fb6c-424b-b80d-bde2cdff4768 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:34.143124 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2637d-fb6c-424b-b80d-bde2cdff4768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:34.197012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-58e2637d-fb6c-424b-b80d-bde2cdff4768 None None] MFA Rules not processed for user `bad955c1488c4c3380db7e84403253ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:34.198370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-58e2637d-fb6c-424b-b80d-bde2cdff4768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:34.259016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:34.295257 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 689/1388] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:34.306922 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0abc0-41f6-4904-82ba-8a7d15d25cab None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:34.307290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0abc0-41f6-4904-82ba-8a7d15d25cab None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:34.307509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0abc0-41f6-4904-82ba-8a7d15d25cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:34.353490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5da0abc0-41f6-4904-82ba-8a7d15d25cab None None] MFA Rules not processed for user `bad955c1488c4c3380db7e84403253ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:34.355526 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5da0abc0-41f6-4904-82ba-8a7d15d25cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:34.387927 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 700/1389] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:34.400463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2637d-fb6c-424b-b80d-bde2cdff4768 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:34.402397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-58e2637d-fb6c-424b-b80d-bde2cdff4768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:34.559378 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2637d-fb6c-424b-b80d-bde2cdff4768 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:34.571821 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89127f01-0d23-4bc8-8e91-9a22a8acddf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:34.572711 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89127f01-0d23-4bc8-8e91-9a22a8acddf4 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:34.573137 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89127f01-0d23-4bc8-8e91-9a22a8acddf4 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:34.573584 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89127f01-0d23-4bc8-8e91-9a22a8acddf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:34.585249 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89127f01-0d23-4bc8-8e91-9a22a8acddf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:34.587262 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89127f01-0d23-4bc8-8e91-9a22a8acddf4 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:34.599177 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 690/1390] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:35.075827 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0abc0-41f6-4904-82ba-8a7d15d25cab None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:35.251272 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0abc0-41f6-4904-82ba-8a7d15d25cab None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:35.258713 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb53180-d2c8-49f0-9699-6172b5ddacac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:35.259728 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb53180-d2c8-49f0-9699-6172b5ddacac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:35.260257 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb53180-d2c8-49f0-9699-6172b5ddacac admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:35.260765 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb53180-d2c8-49f0-9699-6172b5ddacac admin admin] PATH_INFO: `/v3/users/fdb61cf00ee2481193bebea2e3e20c25` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:35.262163 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb53180-d2c8-49f0-9699-6172b5ddacac admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdb61cf00ee2481193bebea2e3e20c25)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:35.265856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb53180-d2c8-49f0-9699-6172b5ddacac admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:35.384100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2fb53180-d2c8-49f0-9699-6172b5ddacac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdb61cf00ee2481193bebea2e3e20c25'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:35.387685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ee618-9bd2-4262-b8b7-095acda38564 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:35.387980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ee618-9bd2-4262-b8b7-095acda38564 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:35.388266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ee618-9bd2-4262-b8b7-095acda38564 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:35.390738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 691/1391] 10.222.0.18 () {54 vars in 912 bytes} [Fri Oct 14 02:24:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:24:35.400469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c195c-38fc-4417-8bc1-829781a1d370 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:35.401002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c195c-38fc-4417-8bc1-829781a1d370 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:35.401525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c195c-38fc-4417-8bc1-829781a1d370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:35.405051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2fb53180-d2c8-49f0-9699-6172b5ddacac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdb61cf00ee2481193bebea2e3e20c25'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:35.424559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:35.435201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2fb53180-d2c8-49f0-9699-6172b5ddacac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdb61cf00ee2481193bebea2e3e20c25'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:35.458321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2fb53180-d2c8-49f0-9699-6172b5ddacac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdb61cf00ee2481193bebea2e3e20c25'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:35.462981 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2fb53180-d2c8-49f0-9699-6172b5ddacac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdb61cf00ee2481193bebea2e3e20c25'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:35.473189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 701/1392] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:35 2022] DELETE /identity/v3/users/fdb61cf00ee2481193bebea2e3e20c25 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:35.478072 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2b09e-ff75-4e18-aa24-6295b0c52166 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:35.479192 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2b09e-ff75-4e18-aa24-6295b0c52166 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:35.479192 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2b09e-ff75-4e18-aa24-6295b0c52166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:35.521127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ff7c195c-38fc-4417-8bc1-829781a1d370 None None] MFA Rules not processed for user `6ff451e43cea4bb0a5fd73491a05bf82`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:35.527900 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ff7c195c-38fc-4417-8bc1-829781a1d370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:35.543266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-dac2b09e-ff75-4e18-aa24-6295b0c52166 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:35.548244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-dac2b09e-ff75-4e18-aa24-6295b0c52166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:35.694599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 692/1393] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:24:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:35.718096 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 702/1394] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:24:35.760353 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c195c-38fc-4417-8bc1-829781a1d370 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:35.765481 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ff7c195c-38fc-4417-8bc1-829781a1d370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:35.766556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2b09e-ff75-4e18-aa24-6295b0c52166 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:35.768702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-dac2b09e-ff75-4e18-aa24-6295b0c52166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:35.787368 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2b09e-ff75-4e18-aa24-6295b0c52166 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:35.790440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c195c-38fc-4417-8bc1-829781a1d370 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:35.795442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8da2a-6a27-4796-854a-33dc8f3603be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:35.796386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8da2a-6a27-4796-854a-33dc8f3603be service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:35.796885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8da2a-6a27-4796-854a-33dc8f3603be service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:35.797350 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8da2a-6a27-4796-854a-33dc8f3603be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:35.801362 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604ba82-dd90-46d3-8ae1-09c887064b06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:35.802163 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604ba82-dd90-46d3-8ae1-09c887064b06 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:35.802461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604ba82-dd90-46d3-8ae1-09c887064b06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:35.802760 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604ba82-dd90-46d3-8ae1-09c887064b06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:35.811895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8da2a-6a27-4796-854a-33dc8f3603be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:35.813755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8da2a-6a27-4796-854a-33dc8f3603be service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:35.823975 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604ba82-dd90-46d3-8ae1-09c887064b06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:35.825447 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604ba82-dd90-46d3-8ae1-09c887064b06 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:35.891889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 703/1396] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:24:35.895060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 693/1395] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:36.204684 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d2601-336a-4a80-8e75-bca9207c3094 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:36.205220 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d2601-336a-4a80-8e75-bca9207c3094 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:36.205383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d2601-336a-4a80-8e75-bca9207c3094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:36.221083 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8da2a-6a27-4796-854a-33dc8f3603be service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:36.223915 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2cd8da2a-6a27-4796-854a-33dc8f3603be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:36.256245 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8da2a-6a27-4796-854a-33dc8f3603be service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:36.277737 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b09d2601-336a-4a80-8e75-bca9207c3094 None None] MFA Rules not processed for user `7d4748cbe63d490b9d8b6914a27ff364`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:36.278983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b09d2601-336a-4a80-8e75-bca9207c3094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:36.283994 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205ea42-2a6e-4bff-ba9b-35da89981e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:36.284937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205ea42-2a6e-4bff-ba9b-35da89981e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:36.285411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205ea42-2a6e-4bff-ba9b-35da89981e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:36.288023 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205ea42-2a6e-4bff-ba9b-35da89981e28 admin admin] PATH_INFO: `/v3/users/4587d8f0deae42f6a64ba824bc720e0e` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:36.291150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205ea42-2a6e-4bff-ba9b-35da89981e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4587d8f0deae42f6a64ba824bc720e0e)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:36.295710 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205ea42-2a6e-4bff-ba9b-35da89981e28 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:36.453224 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f205ea42-2a6e-4bff-ba9b-35da89981e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4587d8f0deae42f6a64ba824bc720e0e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:36.456989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:36.476086 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f205ea42-2a6e-4bff-ba9b-35da89981e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4587d8f0deae42f6a64ba824bc720e0e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:36.476806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f205ea42-2a6e-4bff-ba9b-35da89981e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4587d8f0deae42f6a64ba824bc720e0e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:36.519794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f205ea42-2a6e-4bff-ba9b-35da89981e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4587d8f0deae42f6a64ba824bc720e0e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:36.539536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f205ea42-2a6e-4bff-ba9b-35da89981e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4587d8f0deae42f6a64ba824bc720e0e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:36.557979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 694/1397] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:36 2022] DELETE /identity/v3/users/4587d8f0deae42f6a64ba824bc720e0e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:36.562407 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13e09c-16b8-4ebe-b78d-f30ce634d6d5 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:36.562701 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13e09c-16b8-4ebe-b78d-f30ce634d6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:36.562988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13e09c-16b8-4ebe-b78d-f30ce634d6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:36.585684 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 704/1398] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:36.603733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d2601-336a-4a80-8e75-bca9207c3094 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:36.605309 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b09d2601-336a-4a80-8e75-bca9207c3094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:36.640269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-de13e09c-16b8-4ebe-b78d-f30ce634d6d5 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:36.641792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-de13e09c-16b8-4ebe-b78d-f30ce634d6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:36.783925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 695/1399] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:36.796002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13e09c-16b8-4ebe-b78d-f30ce634d6d5 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:36.797627 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-de13e09c-16b8-4ebe-b78d-f30ce634d6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:36.844305 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:36.918134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d2601-336a-4a80-8e75-bca9207c3094 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:36.974684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13e09c-16b8-4ebe-b78d-f30ce634d6d5 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:36.995608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d050aa6-02be-49da-95cc-81d23f631f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:36.996526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d050aa6-02be-49da-95cc-81d23f631f68 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:36.996889 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d050aa6-02be-49da-95cc-81d23f631f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:36.997618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d050aa6-02be-49da-95cc-81d23f631f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:37.030542 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d050aa6-02be-49da-95cc-81d23f631f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:37.032208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d050aa6-02be-49da-95cc-81d23f631f68 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:37.080190 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 696/1400] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:24:37.081112 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d050aa6-02be-49da-95cc-81d23f631f68 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:37.087514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2d050aa6-02be-49da-95cc-81d23f631f68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:37.102546 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba050862-68ca-4227-b29c-9132c3624bd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:37.106443 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba050862-68ca-4227-b29c-9132c3624bd2 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:37.109914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba050862-68ca-4227-b29c-9132c3624bd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:37.110082 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba050862-68ca-4227-b29c-9132c3624bd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:37.124319 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d050aa6-02be-49da-95cc-81d23f631f68 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:37.142353 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:37.143018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:37.143328 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:37.143616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] PATH_INFO: `/v3/projects/ed9569e193e74dbba29d3b0db1ffe0a1` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:37.145587 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed9569e193e74dbba29d3b0db1ffe0a1)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:37.147613 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:37.342812 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba050862-68ca-4227-b29c-9132c3624bd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:37.346122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba050862-68ca-4227-b29c-9132c3624bd2 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:37.376933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed9569e193e74dbba29d3b0db1ffe0a1', 'request_id': 'req-f7cbb3df-22f3-4640-8d75-26000713c406'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:37.404876 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 705/1401] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 802 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:24:37.422092 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed9569e193e74dbba29d3b0db1ffe0a1', 'request_id': 'req-f7cbb3df-22f3-4640-8d75-26000713c406'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:37.441617 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed9569e193e74dbba29d3b0db1ffe0a1', 'request_id': 'req-f7cbb3df-22f3-4640-8d75-26000713c406'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:37.459726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:37.466023 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba050862-68ca-4227-b29c-9132c3624bd2 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:37.468070 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ba050862-68ca-4227-b29c-9132c3624bd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:37.480640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] The token cache is being invalidate because project ed9569e193e74dbba29d3b0db1ffe0a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:37.480948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:37.485652 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:37.495338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 697/1402] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:37 2022] DELETE /identity/v3/projects/ed9569e193e74dbba29d3b0db1ffe0a1 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:37.496482 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:37.500241 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:37.687599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbb3df-22f3-4640-8d75-26000713c406 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:37.705559 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9618f64-e088-4e8a-845d-16f45c4ab61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:37.706245 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9618f64-e088-4e8a-845d-16f45c4ab61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:37.706582 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9618f64-e088-4e8a-845d-16f45c4ab61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:37.706865 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9618f64-e088-4e8a-845d-16f45c4ab61d admin admin] PATH_INFO: `/v3/users/f01cf12cf3e14a7b9eaeb9d4bb83289d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:37.711109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9618f64-e088-4e8a-845d-16f45c4ab61d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f01cf12cf3e14a7b9eaeb9d4bb83289d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:37.718779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9618f64-e088-4e8a-845d-16f45c4ab61d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:37.751809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba050862-68ca-4227-b29c-9132c3624bd2 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:37.839293 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d9618f64-e088-4e8a-845d-16f45c4ab61d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f01cf12cf3e14a7b9eaeb9d4bb83289d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:37.851377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d9618f64-e088-4e8a-845d-16f45c4ab61d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f01cf12cf3e14a7b9eaeb9d4bb83289d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:37.853021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d9618f64-e088-4e8a-845d-16f45c4ab61d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f01cf12cf3e14a7b9eaeb9d4bb83289d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:37.871782 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:37.882816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d9618f64-e088-4e8a-845d-16f45c4ab61d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f01cf12cf3e14a7b9eaeb9d4bb83289d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:37.895004 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d9618f64-e088-4e8a-845d-16f45c4ab61d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f01cf12cf3e14a7b9eaeb9d4bb83289d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:37.909589 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 698/1403] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:37 2022] DELETE /identity/v3/users/f01cf12cf3e14a7b9eaeb9d4bb83289d => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:37.950076 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3c3a7-5043-4bcf-b58e-84d609115c29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e935ba63cdd4256874b99a2eb0a1a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:37.950772 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3c3a7-5043-4bcf-b58e-84d609115c29 service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:37.950995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3c3a7-5043-4bcf-b58e-84d609115c29 service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:37.951299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3c3a7-5043-4bcf-b58e-84d609115c29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:37.957460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0ad3c3a7-5043-4bcf-b58e-84d609115c29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:37.976466 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3c3a7-5043-4bcf-b58e-84d609115c29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:37.981411 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3c3a7-5043-4bcf-b58e-84d609115c29 service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:38.019584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 706/1404] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:24:38.268695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9618f64-e088-4e8a-845d-16f45c4ab61d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:38.271597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d9618f64-e088-4e8a-845d-16f45c4ab61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:38.529475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9618f64-e088-4e8a-845d-16f45c4ab61d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:38.535275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:38.536882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:38.538274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:38.538759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] PATH_INFO: `/v3/projects/e6596d5252d94376877fbccbe8918ca6` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:38.540918 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6596d5252d94376877fbccbe8918ca6)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:38.544178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:38.572784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:38.709262 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6596d5252d94376877fbccbe8918ca6', 'request_id': 'req-26cbf71f-5ae8-4e97-b775-12d6d681535c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:38.732096 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6596d5252d94376877fbccbe8918ca6', 'request_id': 'req-26cbf71f-5ae8-4e97-b775-12d6d681535c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:38.747833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6596d5252d94376877fbccbe8918ca6', 'request_id': 'req-26cbf71f-5ae8-4e97-b775-12d6d681535c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:38.817585 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] The token cache is being invalidate because project e6596d5252d94376877fbccbe8918ca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:38.821989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:38.825773 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:38.833431 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 699/1405] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:38 2022] DELETE /identity/v3/projects/e6596d5252d94376877fbccbe8918ca6 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:38.837003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3c3a7-5043-4bcf-b58e-84d609115c29 service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:38.863147 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3c3a7-5043-4bcf-b58e-84d609115c29 service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:38.953251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:38.954850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:39.108556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210c7dd-0bd8-4eef-9fd9-719837e014d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:39.109432 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210c7dd-0bd8-4eef-9fd9-719837e014d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:39.110657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210c7dd-0bd8-4eef-9fd9-719837e014d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:39.111091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210c7dd-0bd8-4eef-9fd9-719837e014d8 admin admin] PATH_INFO: `/v3/users/2577d4b0bfab4308b2869349c21fc1fa` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:39.113998 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1210c7dd-0bd8-4eef-9fd9-719837e014d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2577d4b0bfab4308b2869349c21fc1fa)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:39.117824 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1210c7dd-0bd8-4eef-9fd9-719837e014d8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:39.178830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbf71f-5ae8-4e97-b775-12d6d681535c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:39.195618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24fd78-7729-4365-ab4d-422afb7edb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:39.196294 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24fd78-7729-4365-ab4d-422afb7edb74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:39.196576 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24fd78-7729-4365-ab4d-422afb7edb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:39.196853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24fd78-7729-4365-ab4d-422afb7edb74 admin admin] PATH_INFO: `/v3/users/59b2758b1f7e477aa7db11bb1951497d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:39.198082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24fd78-7729-4365-ab4d-422afb7edb74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59b2758b1f7e477aa7db11bb1951497d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:39.200112 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24fd78-7729-4365-ab4d-422afb7edb74 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:39.262804 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1210c7dd-0bd8-4eef-9fd9-719837e014d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2577d4b0bfab4308b2869349c21fc1fa'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:39.287667 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1210c7dd-0bd8-4eef-9fd9-719837e014d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2577d4b0bfab4308b2869349c21fc1fa'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:39.299557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fa24fd78-7729-4365-ab4d-422afb7edb74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59b2758b1f7e477aa7db11bb1951497d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:39.319085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fa24fd78-7729-4365-ab4d-422afb7edb74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59b2758b1f7e477aa7db11bb1951497d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:39.319785 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fa24fd78-7729-4365-ab4d-422afb7edb74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59b2758b1f7e477aa7db11bb1951497d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:39.342475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fa24fd78-7729-4365-ab4d-422afb7edb74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59b2758b1f7e477aa7db11bb1951497d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:39.344694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1210c7dd-0bd8-4eef-9fd9-719837e014d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2577d4b0bfab4308b2869349c21fc1fa'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:39.354823 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1210c7dd-0bd8-4eef-9fd9-719837e014d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2577d4b0bfab4308b2869349c21fc1fa'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:39.355813 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1210c7dd-0bd8-4eef-9fd9-719837e014d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2577d4b0bfab4308b2869349c21fc1fa'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:39.359927 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fa24fd78-7729-4365-ab4d-422afb7edb74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59b2758b1f7e477aa7db11bb1951497d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:39.366549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 707/1406] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:38 2022] DELETE /identity/v3/users/2577d4b0bfab4308b2869349c21fc1fa => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:39.386868 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 700/1407] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:38 2022] DELETE /identity/v3/users/59b2758b1f7e477aa7db11bb1951497d => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:24:39.390720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b8235-01c2-4d47-965b-3fb649360183 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:39.391377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b8235-01c2-4d47-965b-3fb649360183 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:39.392074 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b8235-01c2-4d47-965b-3fb649360183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:39.425447 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d90b8235-01c2-4d47-965b-3fb649360183 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:39.427453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d90b8235-01c2-4d47-965b-3fb649360183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:39.589199 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:39.599345 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24fd78-7729-4365-ab4d-422afb7edb74 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:39.603018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-fa24fd78-7729-4365-ab4d-422afb7edb74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:39.657007 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 708/1408] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:39.686555 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b8235-01c2-4d47-965b-3fb649360183 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:39.688288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d90b8235-01c2-4d47-965b-3fb649360183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:39.715453 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:39.831128 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24fd78-7729-4365-ab4d-422afb7edb74 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:39.850193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5711252-18bb-4931-b688-f55ef0d8665f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:39.850852 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5711252-18bb-4931-b688-f55ef0d8665f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:39.851151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5711252-18bb-4931-b688-f55ef0d8665f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:39.851425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5711252-18bb-4931-b688-f55ef0d8665f admin admin] PATH_INFO: `/v3/projects/284bb33fe40444238f85b8d689ef46e2` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:39.852905 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5711252-18bb-4931-b688-f55ef0d8665f admin admin] RBAC: Authorizing `identity:delete_project(project_id=284bb33fe40444238f85b8d689ef46e2)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:39.854890 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5711252-18bb-4931-b688-f55ef0d8665f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:39.912582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b8235-01c2-4d47-965b-3fb649360183 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:39.923773 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769e1ba-0c28-4319-863d-4c4cd99da688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:39.924416 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769e1ba-0c28-4319-863d-4c4cd99da688 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:39.924696 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769e1ba-0c28-4319-863d-4c4cd99da688 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:39.924970 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769e1ba-0c28-4319-863d-4c4cd99da688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:39.978722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769e1ba-0c28-4319-863d-4c4cd99da688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:39.981106 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769e1ba-0c28-4319-863d-4c4cd99da688 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:40.000755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c5711252-18bb-4931-b688-f55ef0d8665f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '284bb33fe40444238f85b8d689ef46e2', 'request_id': 'req-c5711252-18bb-4931-b688-f55ef0d8665f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:40.009802 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 709/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:24:40.011216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada41aa-8ab1-45b9-bcbb-fd387f315db9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:40.011506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada41aa-8ab1-45b9-bcbb-fd387f315db9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:40.011783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada41aa-8ab1-45b9-bcbb-fd387f315db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:40.020346 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c5711252-18bb-4931-b688-f55ef0d8665f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '284bb33fe40444238f85b8d689ef46e2', 'request_id': 'req-c5711252-18bb-4931-b688-f55ef0d8665f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:40.039875 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c5711252-18bb-4931-b688-f55ef0d8665f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '284bb33fe40444238f85b8d689ef46e2', 'request_id': 'req-c5711252-18bb-4931-b688-f55ef0d8665f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:40.074704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-eada41aa-8ab1-45b9-bcbb-fd387f315db9 None None] MFA Rules not processed for user `e4ebab767c054b289e2279659a1cf2e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:40.082110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-eada41aa-8ab1-45b9-bcbb-fd387f315db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:40.093625 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c5711252-18bb-4931-b688-f55ef0d8665f admin admin] The token cache is being invalidate because project 284bb33fe40444238f85b8d689ef46e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:40.093943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c5711252-18bb-4931-b688-f55ef0d8665f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:40.094888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c5711252-18bb-4931-b688-f55ef0d8665f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:40.100664 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 701/1410] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:39 2022] DELETE /identity/v3/projects/284bb33fe40444238f85b8d689ef46e2 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:40.105474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e63103-a1b7-4613-8a7d-e5327a7ed465 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:40.105474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e63103-a1b7-4613-8a7d-e5327a7ed465 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:40.105474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e63103-a1b7-4613-8a7d-e5327a7ed465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:40.199009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b7e63103-a1b7-4613-8a7d-e5327a7ed465 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:40.200281 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b7e63103-a1b7-4613-8a7d-e5327a7ed465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:40.402548 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 710/1411] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:40.409741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada41aa-8ab1-45b9-bcbb-fd387f315db9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:40.413873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-eada41aa-8ab1-45b9-bcbb-fd387f315db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:40.489733 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 702/1412] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:40.494020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022765a-4535-4da0-bb42-db14eaa78fd7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:40.494657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022765a-4535-4da0-bb42-db14eaa78fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:40.494657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022765a-4535-4da0-bb42-db14eaa78fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:40.584822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d022765a-4535-4da0-bb42-db14eaa78fd7 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:40.586045 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d022765a-4535-4da0-bb42-db14eaa78fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:40.610944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 703/1413] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:40.612282 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022765a-4535-4da0-bb42-db14eaa78fd7 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:40.634282 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022765a-4535-4da0-bb42-db14eaa78fd7 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:40.634670 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada41aa-8ab1-45b9-bcbb-fd387f315db9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:40.650048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cc93b-b908-4511-bf2c-2083d64622c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:40.650935 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cc93b-b908-4511-bf2c-2083d64622c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:40.651689 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cc93b-b908-4511-bf2c-2083d64622c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:40.652156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cc93b-b908-4511-bf2c-2083d64622c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:40.653047 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7cc93b-b908-4511-bf2c-2083d64622c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:40.653564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7cc93b-b908-4511-bf2c-2083d64622c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:40.655513 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7cc93b-b908-4511-bf2c-2083d64622c2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:40.657220 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0a0b5-5f34-4222-af00-7a6f73cf1fe2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:40.658024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0a0b5-5f34-4222-af00-7a6f73cf1fe2 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:40.658024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0a0b5-5f34-4222-af00-7a6f73cf1fe2 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:40.658293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0a0b5-5f34-4222-af00-7a6f73cf1fe2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:40.681556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0a0b5-5f34-4222-af00-7a6f73cf1fe2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:40.683312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0a0b5-5f34-4222-af00-7a6f73cf1fe2 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:40.698169 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 704/1414] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:40.699624 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cc93b-b908-4511-bf2c-2083d64622c2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:40.728005 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 711/1415] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:24:40.730070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cc93b-b908-4511-bf2c-2083d64622c2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:40.745386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27357a7-b749-40c0-a952-66df385b5273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:40.746254 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27357a7-b749-40c0-a952-66df385b5273 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:40.746640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27357a7-b749-40c0-a952-66df385b5273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:40.747016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27357a7-b749-40c0-a952-66df385b5273 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:40.747822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27357a7-b749-40c0-a952-66df385b5273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:40.750364 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27357a7-b749-40c0-a952-66df385b5273 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:40.796248 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0a0b5-5f34-4222-af00-7a6f73cf1fe2 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:40.798032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-80f0a0b5-5f34-4222-af00-7a6f73cf1fe2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:40.820943 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:40.830217 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 705/1416] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:40 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:40.839874 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27357a7-b749-40c0-a952-66df385b5273 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:40.843919 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0a0b5-5f34-4222-af00-7a6f73cf1fe2 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:40.889951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:41.053735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0925c11-3880-47dc-aa2f-13daa73e3ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:41.055605 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0925c11-3880-47dc-aa2f-13daa73e3ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:41.056928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0925c11-3880-47dc-aa2f-13daa73e3ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:41.057618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27357a7-b749-40c0-a952-66df385b5273 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:41.064063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0925c11-3880-47dc-aa2f-13daa73e3ef8 admin admin] PATH_INFO: `/v3/projects/587d9c00612f44498152cb13f5d1fe12` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:41.067313 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0925c11-3880-47dc-aa2f-13daa73e3ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=587d9c00612f44498152cb13f5d1fe12)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:41.069913 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0925c11-3880-47dc-aa2f-13daa73e3ef8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:41.073680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fed903-79e4-45ab-8637-b966e16990a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:41.074876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fed903-79e4-45ab-8637-b966e16990a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:41.075349 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fed903-79e4-45ab-8637-b966e16990a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:41.075768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fed903-79e4-45ab-8637-b966e16990a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:41.076570 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fed903-79e4-45ab-8637-b966e16990a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:41.078747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fed903-79e4-45ab-8637-b966e16990a4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:41.130551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 706/1417] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:40 2022] POST /identity/v3/users => generated 294 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:41.135046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fed903-79e4-45ab-8637-b966e16990a4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:41.155870 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fed903-79e4-45ab-8637-b966e16990a4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:41.182371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d0925c11-3880-47dc-aa2f-13daa73e3ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '587d9c00612f44498152cb13f5d1fe12', 'request_id': 'req-d0925c11-3880-47dc-aa2f-13daa73e3ef8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:41.188606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d0925c11-3880-47dc-aa2f-13daa73e3ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '587d9c00612f44498152cb13f5d1fe12', 'request_id': 'req-d0925c11-3880-47dc-aa2f-13daa73e3ef8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:41.200293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d0925c11-3880-47dc-aa2f-13daa73e3ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '587d9c00612f44498152cb13f5d1fe12', 'request_id': 'req-d0925c11-3880-47dc-aa2f-13daa73e3ef8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:41.228619 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d0925c11-3880-47dc-aa2f-13daa73e3ef8 admin admin] The token cache is being invalidate because project 587d9c00612f44498152cb13f5d1fe12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:41.229170 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d0925c11-3880-47dc-aa2f-13daa73e3ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:41.229961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d0925c11-3880-47dc-aa2f-13daa73e3ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:41.231954 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 712/1418] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:40 2022] DELETE /identity/v3/projects/587d9c00612f44498152cb13f5d1fe12 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:41.240433 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b2943-bd53-4e09-9412-d39624b52e9c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:41.241165 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b2943-bd53-4e09-9412-d39624b52e9c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:41.241605 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b2943-bd53-4e09-9412-d39624b52e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:41.277231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bb319-81bc-4660-b511-45f315e59ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:41.277902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bb319-81bc-4660-b511-45f315e59ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:41.278182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bb319-81bc-4660-b511-45f315e59ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:41.278460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bb319-81bc-4660-b511-45f315e59ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:41.279144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bb319-81bc-4660-b511-45f315e59ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:41.282200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bb319-81bc-4660-b511-45f315e59ba1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:41.298227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-195b2943-bd53-4e09-9412-d39624b52e9c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:41.304436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-195b2943-bd53-4e09-9412-d39624b52e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:41.327120 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 707/1419] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:41.336657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc206307-122e-48c2-8333-77eb042b4186 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:41.337000 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc206307-122e-48c2-8333-77eb042b4186 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:41.337899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc206307-122e-48c2-8333-77eb042b4186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:41.384571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 713/1420] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:41.385533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b2943-bd53-4e09-9412-d39624b52e9c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:41.390678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-195b2943-bd53-4e09-9412-d39624b52e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:41.419369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-dc206307-122e-48c2-8333-77eb042b4186 None None] MFA Rules not processed for user `79365a5e4ae2431d91a4aa603ba58fee`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:41.420666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dc206307-122e-48c2-8333-77eb042b4186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:41.429792 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b2943-bd53-4e09-9412-d39624b52e9c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:41.444943 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85509a65-d904-4fc7-9bf6-6dd24cd5f928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:41.445757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85509a65-d904-4fc7-9bf6-6dd24cd5f928 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:41.445757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85509a65-d904-4fc7-9bf6-6dd24cd5f928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:41.446021 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85509a65-d904-4fc7-9bf6-6dd24cd5f928 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:41.446684 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85509a65-d904-4fc7-9bf6-6dd24cd5f928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:41.448336 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85509a65-d904-4fc7-9bf6-6dd24cd5f928 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:41.507731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 714/1421] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:41.509107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eeb86-b9df-4a6f-a7c3-c432b3550c5d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:41.511273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eeb86-b9df-4a6f-a7c3-c432b3550c5d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:41.511916 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eeb86-b9df-4a6f-a7c3-c432b3550c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:41.601075 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-327eeb86-b9df-4a6f-a7c3-c432b3550c5d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:41.602312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-327eeb86-b9df-4a6f-a7c3-c432b3550c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:41.619695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 708/1422] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:24:41.620801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc206307-122e-48c2-8333-77eb042b4186 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:41.666138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 715/1423] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:41.670880 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64ba6d-1999-4af7-ad4d-09afb30b1687 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:41.672658 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64ba6d-1999-4af7-ad4d-09afb30b1687 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:41.676025 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc206307-122e-48c2-8333-77eb042b4186 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:41.678305 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64ba6d-1999-4af7-ad4d-09afb30b1687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:41.707720 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afd938-6e70-4cb2-9dda-0f469bdc23b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:41.708698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afd938-6e70-4cb2-9dda-0f469bdc23b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:41.709722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afd938-6e70-4cb2-9dda-0f469bdc23b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:41.710002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afd938-6e70-4cb2-9dda-0f469bdc23b0 admin admin] PATH_INFO: `/v3/projects/dcbc1920380a46b0a66704f886bf9c18/users/7241784ff34e40458e0d3f214aac32e5/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:41.766622 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ec64ba6d-1999-4af7-ad4d-09afb30b1687 None None] MFA Rules not processed for user `79365a5e4ae2431d91a4aa603ba58fee`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:41.770538 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ec64ba6d-1999-4af7-ad4d-09afb30b1687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:41.772903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afd938-6e70-4cb2-9dda-0f469bdc23b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcbc1920380a46b0a66704f886bf9c18, user_id=7241784ff34e40458e0d3f214aac32e5, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:41.775804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afd938-6e70-4cb2-9dda-0f469bdc23b0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:41.808292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 716/1424] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:41.809689 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64ba6d-1999-4af7-ad4d-09afb30b1687 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:41.818847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 709/1425] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:41 2022] PUT /identity/v3/projects/dcbc1920380a46b0a66704f886bf9c18/users/7241784ff34e40458e0d3f214aac32e5/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:41.826040 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afd938-6e70-4cb2-9dda-0f469bdc23b0 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:41.838025 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-75afd938-6e70-4cb2-9dda-0f469bdc23b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:41.854867 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:41.867100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64ba6d-1999-4af7-ad4d-09afb30b1687 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:42.026758 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831fbf9-d2b7-4fd3-b016-811554553727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:42.028119 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831fbf9-d2b7-4fd3-b016-811554553727 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:42.028575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831fbf9-d2b7-4fd3-b016-811554553727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:42.028870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831fbf9-d2b7-4fd3-b016-811554553727 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:42.029547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831fbf9-d2b7-4fd3-b016-811554553727 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:42.029867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831fbf9-d2b7-4fd3-b016-811554553727 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:42.033516 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831fbf9-d2b7-4fd3-b016-811554553727 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:42.054292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:42.066532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 717/1426] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:42.073447 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c12ef-1b1f-466e-833d-134f266b9d56 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:42.075249 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c12ef-1b1f-466e-833d-134f266b9d56 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:42.075631 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c12ef-1b1f-466e-833d-134f266b9d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:42.107871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afd938-6e70-4cb2-9dda-0f469bdc23b0 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:42.118558 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fceac48-6242-44db-8a1a-b40d1e1da55a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:42.120071 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fceac48-6242-44db-8a1a-b40d1e1da55a service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:42.120071 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fceac48-6242-44db-8a1a-b40d1e1da55a service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:42.120071 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fceac48-6242-44db-8a1a-b40d1e1da55a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:42.148989 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d77c12ef-1b1f-466e-833d-134f266b9d56 None None] MFA Rules not processed for user `7241784ff34e40458e0d3f214aac32e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:42.150574 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d77c12ef-1b1f-466e-833d-134f266b9d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:42.291409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fceac48-6242-44db-8a1a-b40d1e1da55a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:42.296571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 718/1427] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:42.297337 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fceac48-6242-44db-8a1a-b40d1e1da55a service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:42.297581 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c12ef-1b1f-466e-833d-134f266b9d56 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:42.302384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d77c12ef-1b1f-466e-833d-134f266b9d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:42.329388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 710/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:42.330464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fceac48-6242-44db-8a1a-b40d1e1da55a service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:42.333943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3fceac48-6242-44db-8a1a-b40d1e1da55a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:42.348965 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c12ef-1b1f-466e-833d-134f266b9d56 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:42.367674 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:42.368606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:42.368800 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:42.369078 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] PATH_INFO: `/v3/projects/efb5f401afb04c6ca3fdd2c437174615` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:42.372451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efb5f401afb04c6ca3fdd2c437174615)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:42.374516 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:42.387047 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fceac48-6242-44db-8a1a-b40d1e1da55a service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:42.398642 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82744eb-f846-4d82-b973-f3e172a9b011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:42.399684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82744eb-f846-4d82-b973-f3e172a9b011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:42.400346 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82744eb-f846-4d82-b973-f3e172a9b011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:42.400860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82744eb-f846-4d82-b973-f3e172a9b011 admin admin] PATH_INFO: `/v3/users/9be053db30cd49eeba69b4d3a923cff0` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:42.402935 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82744eb-f846-4d82-b973-f3e172a9b011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9be053db30cd49eeba69b4d3a923cff0)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:42.405364 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82744eb-f846-4d82-b973-f3e172a9b011 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:42.508077 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efb5f401afb04c6ca3fdd2c437174615', 'request_id': 'req-1121b27d-756d-4b44-b2d1-48cfff05b5b0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:42.524695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efb5f401afb04c6ca3fdd2c437174615', 'request_id': 'req-1121b27d-756d-4b44-b2d1-48cfff05b5b0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:42.532384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efb5f401afb04c6ca3fdd2c437174615', 'request_id': 'req-1121b27d-756d-4b44-b2d1-48cfff05b5b0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:42.563320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] The token cache is being invalidate because project efb5f401afb04c6ca3fdd2c437174615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:42.563639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:42.567831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:42.569726 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 719/1429] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:42 2022] DELETE /identity/v3/projects/efb5f401afb04c6ca3fdd2c437174615 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:42.570239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:42.573825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:42.587581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f82744eb-f846-4d82-b973-f3e172a9b011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9be053db30cd49eeba69b4d3a923cff0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:42.610854 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f82744eb-f846-4d82-b973-f3e172a9b011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9be053db30cd49eeba69b4d3a923cff0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:42.612486 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f82744eb-f846-4d82-b973-f3e172a9b011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9be053db30cd49eeba69b4d3a923cff0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:42.653071 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f82744eb-f846-4d82-b973-f3e172a9b011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9be053db30cd49eeba69b4d3a923cff0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:42.695845 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f82744eb-f846-4d82-b973-f3e172a9b011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9be053db30cd49eeba69b4d3a923cff0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:42.714537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 711/1430] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:42 2022] DELETE /identity/v3/users/9be053db30cd49eeba69b4d3a923cff0 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:42.715752 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82744eb-f846-4d82-b973-f3e172a9b011 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:42.717291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f82744eb-f846-4d82-b973-f3e172a9b011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:42.726634 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121b27d-756d-4b44-b2d1-48cfff05b5b0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:42.744827 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:42.754812 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:42.878188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82744eb-f846-4d82-b973-f3e172a9b011 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:42.886210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983a24b-8dd9-40b2-93ae-78924a0b33a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:42.887820 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983a24b-8dd9-40b2-93ae-78924a0b33a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:42.889136 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983a24b-8dd9-40b2-93ae-78924a0b33a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:42.889925 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983a24b-8dd9-40b2-93ae-78924a0b33a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:42.890669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0133e-0572-4de8-8b99-255642053ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:42.892212 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0133e-0572-4de8-8b99-255642053ebd admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:42.892457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983a24b-8dd9-40b2-93ae-78924a0b33a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:42.895721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0133e-0572-4de8-8b99-255642053ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:42.896279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0133e-0572-4de8-8b99-255642053ebd admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:42.897139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0133e-0572-4de8-8b99-255642053ebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:42.898013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983a24b-8dd9-40b2-93ae-78924a0b33a6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:42.902067 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0133e-0572-4de8-8b99-255642053ebd admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:42.990180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 712/1431] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:42 2022] POST /identity/v3/projects => generated 387 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:42.991448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0133e-0572-4de8-8b99-255642053ebd admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:42.992049 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 720/1432] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:42 2022] POST /identity/v3/projects => generated 381 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:42.994478 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285fb98-cd0c-4d6c-b5d6-a0605ff17ee7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:42.994773 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285fb98-cd0c-4d6c-b5d6-a0605ff17ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:42.995162 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285fb98-cd0c-4d6c-b5d6-a0605ff17ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:43.001766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-74a0133e-0572-4de8-8b99-255642053ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:43.133828 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f285fb98-cd0c-4d6c-b5d6-a0605ff17ee7 None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:43.135102 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f285fb98-cd0c-4d6c-b5d6-a0605ff17ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:43.172552 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0133e-0572-4de8-8b99-255642053ebd admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:43.184645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ec339-8963-4eb0-b135-a29a59b02eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:43.185314 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ec339-8963-4eb0-b135-a29a59b02eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:43.185596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ec339-8963-4eb0-b135-a29a59b02eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:43.185872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ec339-8963-4eb0-b135-a29a59b02eea admin admin] PATH_INFO: `/v3/users/d5d60c72d8944e6bb61f01ddacf5864f` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:43.190972 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ec339-8963-4eb0-b135-a29a59b02eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d60c72d8944e6bb61f01ddacf5864f)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:43.193520 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ec339-8963-4eb0-b135-a29a59b02eea admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:43.350040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 721/1433] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:24:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:43.351488 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d4cc2-3ddf-4d82-8817-866f7889967d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:43.352044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d4cc2-3ddf-4d82-8817-866f7889967d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:43.352587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d4cc2-3ddf-4d82-8817-866f7889967d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:43.403400 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-308ec339-8963-4eb0-b135-a29a59b02eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d60c72d8944e6bb61f01ddacf5864f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:43.424936 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-308ec339-8963-4eb0-b135-a29a59b02eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d60c72d8944e6bb61f01ddacf5864f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:43.425555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-308ec339-8963-4eb0-b135-a29a59b02eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d60c72d8944e6bb61f01ddacf5864f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:43.452887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-308ec339-8963-4eb0-b135-a29a59b02eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d60c72d8944e6bb61f01ddacf5864f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:43.461794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-373d4cc2-3ddf-4d82-8817-866f7889967d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:43.462070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-308ec339-8963-4eb0-b135-a29a59b02eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d60c72d8944e6bb61f01ddacf5864f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:43.464089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-373d4cc2-3ddf-4d82-8817-866f7889967d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:43.470588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 713/1434] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:42 2022] DELETE /identity/v3/users/d5d60c72d8944e6bb61f01ddacf5864f => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:43.476392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ec339-8963-4eb0-b135-a29a59b02eea admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:43.478163 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 722/1435] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:43.479251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d4cc2-3ddf-4d82-8817-866f7889967d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:43.480754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-373d4cc2-3ddf-4d82-8817-866f7889967d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:43.482313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-308ec339-8963-4eb0-b135-a29a59b02eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:43.501704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d4cc2-3ddf-4d82-8817-866f7889967d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:43.532629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:43.598656 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ec339-8963-4eb0-b135-a29a59b02eea admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:43.607404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431562f-48a6-431a-9991-3a2e12dab9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:43.608055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431562f-48a6-431a-9991-3a2e12dab9a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:43.608564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431562f-48a6-431a-9991-3a2e12dab9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:43.608841 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431562f-48a6-431a-9991-3a2e12dab9a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:43.609717 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431562f-48a6-431a-9991-3a2e12dab9a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:43.612437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431562f-48a6-431a-9991-3a2e12dab9a8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:43.632885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f1eb3-98e5-4744-b3b8-11d3d3a1c43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:43.634388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f1eb3-98e5-4744-b3b8-11d3d3a1c43a admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:43.635310 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f1eb3-98e5-4744-b3b8-11d3d3a1c43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:43.635891 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f1eb3-98e5-4744-b3b8-11d3d3a1c43a admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:43.636845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f1eb3-98e5-4744-b3b8-11d3d3a1c43a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:43.639452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f1eb3-98e5-4744-b3b8-11d3d3a1c43a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:43.655429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 714/1436] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:43 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:43.656724 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431562f-48a6-431a-9991-3a2e12dab9a8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:43.659369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6431562f-48a6-431a-9991-3a2e12dab9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:43.679573 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:43.693303 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 723/1437] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:43 2022] POST /identity/v3/users => generated 300 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:43.695223 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f1eb3-98e5-4744-b3b8-11d3d3a1c43a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:43.712582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f1eb3-98e5-4744-b3b8-11d3d3a1c43a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:43.731364 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc776fe-1e1c-44c0-8308-8a778d174458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:43.732166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc776fe-1e1c-44c0-8308-8a778d174458 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:43.732577 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc776fe-1e1c-44c0-8308-8a778d174458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:43.732982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc776fe-1e1c-44c0-8308-8a778d174458 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:43.733762 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc776fe-1e1c-44c0-8308-8a778d174458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:43.743782 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431562f-48a6-431a-9991-3a2e12dab9a8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:43.745052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc776fe-1e1c-44c0-8308-8a778d174458 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:43.748596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa60a0-1c7b-41fb-860d-c82686e43107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:43.749300 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa60a0-1c7b-41fb-860d-c82686e43107 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:43.749948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa60a0-1c7b-41fb-860d-c82686e43107 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:43.750239 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa60a0-1c7b-41fb-860d-c82686e43107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:43.765444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa60a0-1c7b-41fb-860d-c82686e43107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:43.766932 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa60a0-1c7b-41fb-860d-c82686e43107 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:43.771192 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 724/1438] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:43.772264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc776fe-1e1c-44c0-8308-8a778d174458 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:43.774998 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-abc776fe-1e1c-44c0-8308-8a778d174458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:43.821104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 715/1439] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:43.824468 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa60a0-1c7b-41fb-860d-c82686e43107 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:43.838412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa60a0-1c7b-41fb-860d-c82686e43107 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:43.845170 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c925f1f-741f-4483-a547-031c3d5bdf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:43.845807 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c925f1f-741f-4483-a547-031c3d5bdf6f admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:43.846242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c925f1f-741f-4483-a547-031c3d5bdf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:43.846830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c925f1f-741f-4483-a547-031c3d5bdf6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:43.847226 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c925f1f-741f-4483-a547-031c3d5bdf6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:43.852866 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c925f1f-741f-4483-a547-031c3d5bdf6f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:43.875031 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 716/1440] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:43.875031 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c925f1f-741f-4483-a547-031c3d5bdf6f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:43.891709 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c925f1f-741f-4483-a547-031c3d5bdf6f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:43.898440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6459cc5-9108-4be4-977b-b87bd8787efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:43.899109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6459cc5-9108-4be4-977b-b87bd8787efc admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:43.899608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6459cc5-9108-4be4-977b-b87bd8787efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:43.899928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6459cc5-9108-4be4-977b-b87bd8787efc admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:43.900933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6459cc5-9108-4be4-977b-b87bd8787efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:43.902791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6459cc5-9108-4be4-977b-b87bd8787efc admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:43.919563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 717/1441] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:43.919563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c63cc1-52ed-4f11-b8e9-ed48ae277f80 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:43.919563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c63cc1-52ed-4f11-b8e9-ed48ae277f80 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:43.919563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c63cc1-52ed-4f11-b8e9-ed48ae277f80 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:43.920021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 718/1442] 10.222.0.18 () {54 vars in 912 bytes} [Fri Oct 14 02:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:24:43.920817 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c63cc1-52ed-4f11-b8e9-ed48ae277f80 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:43.959880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c63cc1-52ed-4f11-b8e9-ed48ae277f80 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:43.965858 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e4bc9-70da-4374-a666-94b6b889ab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:43.966793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e4bc9-70da-4374-a666-94b6b889ab59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:43.967100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e4bc9-70da-4374-a666-94b6b889ab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:43.967549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e4bc9-70da-4374-a666-94b6b889ab59 admin admin] PATH_INFO: `/v3/projects/090b7837445f44699c852450f2e0e8ee/users/0c464e2a20514b55a92936bad246e29b/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:43.989736 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e4bc9-70da-4374-a666-94b6b889ab59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=090b7837445f44699c852450f2e0e8ee, user_id=0c464e2a20514b55a92936bad246e29b, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:44.000822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e4bc9-70da-4374-a666-94b6b889ab59 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:44.012034 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc776fe-1e1c-44c0-8308-8a778d174458 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:44.035884 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 719/1443] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:43 2022] PUT /identity/v3/projects/090b7837445f44699c852450f2e0e8ee/users/0c464e2a20514b55a92936bad246e29b/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:44.035884 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e4bc9-70da-4374-a666-94b6b889ab59 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:44.133978 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9147aa6-5bbb-480c-915b-2020461e9b1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:44.135691 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9147aa6-5bbb-480c-915b-2020461e9b1c service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:44.135977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9147aa6-5bbb-480c-915b-2020461e9b1c service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:44.136254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9147aa6-5bbb-480c-915b-2020461e9b1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:44.138231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b9147aa6-5bbb-480c-915b-2020461e9b1c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:44.203817 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e4bc9-70da-4374-a666-94b6b889ab59 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:44.219699 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ed5e1-2a58-4f38-8f5d-d59bf082cfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:44.219924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ed5e1-2a58-4f38-8f5d-d59bf082cfdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:44.220218 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ed5e1-2a58-4f38-8f5d-d59bf082cfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:44.220501 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ed5e1-2a58-4f38-8f5d-d59bf082cfdf admin admin] PATH_INFO: `/v3/projects/392db12b4b4449f19aeaa8e477ce0b1d/users/16b3f322fd3f4709800c6e7f850d4a84/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:44.226976 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:44.258776 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ed5e1-2a58-4f38-8f5d-d59bf082cfdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=392db12b4b4449f19aeaa8e477ce0b1d, user_id=16b3f322fd3f4709800c6e7f850d4a84, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:44.262701 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ed5e1-2a58-4f38-8f5d-d59bf082cfdf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:44.302826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 720/1444] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:44 2022] PUT /identity/v3/projects/392db12b4b4449f19aeaa8e477ce0b1d/users/16b3f322fd3f4709800c6e7f850d4a84/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:44.302826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03603a00-6fb7-4bf9-808b-26d465a51bf9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:44.302826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03603a00-6fb7-4bf9-808b-26d465a51bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:44.302826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03603a00-6fb7-4bf9-808b-26d465a51bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:44.325232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9147aa6-5bbb-480c-915b-2020461e9b1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:44.342321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9147aa6-5bbb-480c-915b-2020461e9b1c service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:44.358600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-03603a00-6fb7-4bf9-808b-26d465a51bf9 None None] MFA Rules not processed for user `6ff451e43cea4bb0a5fd73491a05bf82`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:44.361346 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-03603a00-6fb7-4bf9-808b-26d465a51bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:44.386236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 725/1445] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:24:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:24:44.390868 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ce624-5b7f-4f0e-84e0-2af5fb9911b6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:44.391477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ce624-5b7f-4f0e-84e0-2af5fb9911b6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:44.391928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ce624-5b7f-4f0e-84e0-2af5fb9911b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:44.493058 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-741ce624-5b7f-4f0e-84e0-2af5fb9911b6 None None] MFA Rules not processed for user `7d4748cbe63d490b9d8b6914a27ff364`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:44.498436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-741ce624-5b7f-4f0e-84e0-2af5fb9911b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:44.522729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:44.621299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 721/1446] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:24:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:44.631178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03603a00-6fb7-4bf9-808b-26d465a51bf9 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:44.798809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 726/1447] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:44.804696 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec7042-5933-464d-979c-74633803bc0d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:44.805806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec7042-5933-464d-979c-74633803bc0d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:44.805806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec7042-5933-464d-979c-74633803bc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:44.882753 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03603a00-6fb7-4bf9-808b-26d465a51bf9 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:44.914204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e321c-c69a-4387-a7f1-ca77666cc369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:44.914874 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e321c-c69a-4387-a7f1-ca77666cc369 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:44.915185 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e321c-c69a-4387-a7f1-ca77666cc369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:44.915464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e321c-c69a-4387-a7f1-ca77666cc369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:44.920924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e321c-c69a-4387-a7f1-ca77666cc369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:44.920924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e321c-c69a-4387-a7f1-ca77666cc369 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:44.921154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9fec7042-5933-464d-979c-74633803bc0d None None] MFA Rules not processed for user `16b3f322fd3f4709800c6e7f850d4a84`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:44.929850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9fec7042-5933-464d-979c-74633803bc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:44.966559 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 722/1448] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:44.968844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e321c-c69a-4387-a7f1-ca77666cc369 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:44.976933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8e6e321c-c69a-4387-a7f1-ca77666cc369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:45.025117 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e321c-c69a-4387-a7f1-ca77666cc369 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:45.032909 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4482554-4731-4ee0-99f6-7f0d7da52823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:45.033898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4482554-4731-4ee0-99f6-7f0d7da52823 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:45.033898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4482554-4731-4ee0-99f6-7f0d7da52823 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:45.034130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4482554-4731-4ee0-99f6-7f0d7da52823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:45.049528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4482554-4731-4ee0-99f6-7f0d7da52823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:45.054983 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4482554-4731-4ee0-99f6-7f0d7da52823 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:45.078223 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 723/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:45.079110 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4482554-4731-4ee0-99f6-7f0d7da52823 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:45.108953 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4482554-4731-4ee0-99f6-7f0d7da52823 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:45.122680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a33dd-991d-4216-96ff-10887fca6ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:45.123484 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a33dd-991d-4216-96ff-10887fca6ccd admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:45.123844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a33dd-991d-4216-96ff-10887fca6ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:45.124128 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a33dd-991d-4216-96ff-10887fca6ccd admin admin] PATH_INFO: `/v3/projects/090b7837445f44699c852450f2e0e8ee/users/0c464e2a20514b55a92936bad246e29b/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:45.138753 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a33dd-991d-4216-96ff-10887fca6ccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=090b7837445f44699c852450f2e0e8ee, user_id=0c464e2a20514b55a92936bad246e29b, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:45.141588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a33dd-991d-4216-96ff-10887fca6ccd admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:45.179622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 724/1450] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:45 2022] PUT /identity/v3/projects/090b7837445f44699c852450f2e0e8ee/users/0c464e2a20514b55a92936bad246e29b/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:45.193502 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d3788-4f6a-4173-8121-4489dcdc85ff None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:45.194427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d3788-4f6a-4173-8121-4489dcdc85ff None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:45.194725 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d3788-4f6a-4173-8121-4489dcdc85ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:45.227965 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 727/1451] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:45.236093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec7042-5933-464d-979c-74633803bc0d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:45.245096 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-936d3788-4f6a-4173-8121-4489dcdc85ff None None] MFA Rules not processed for user `0c464e2a20514b55a92936bad246e29b`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:45.246610 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-936d3788-4f6a-4173-8121-4489dcdc85ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:45.272517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec7042-5933-464d-979c-74633803bc0d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:45.425390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:45.492646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 725/1452] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:45.515896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d3788-4f6a-4173-8121-4489dcdc85ff None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:45.524341 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-936d3788-4f6a-4173-8121-4489dcdc85ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:45.718758 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d3788-4f6a-4173-8121-4489dcdc85ff None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:45.747863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10b1a4-902d-4832-b423-d6f10b9afea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:45.748522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10b1a4-902d-4832-b423-d6f10b9afea4 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:45.748802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10b1a4-902d-4832-b423-d6f10b9afea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:45.749073 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10b1a4-902d-4832-b423-d6f10b9afea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:45.790872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10b1a4-902d-4832-b423-d6f10b9afea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:45.799310 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10b1a4-902d-4832-b423-d6f10b9afea4 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:45.813667 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf58a1-75c1-451c-97ad-7da8011085e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:45.817388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf58a1-75c1-451c-97ad-7da8011085e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:45.817731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf58a1-75c1-451c-97ad-7da8011085e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:45.818016 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf58a1-75c1-451c-97ad-7da8011085e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:45.819477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf58a1-75c1-451c-97ad-7da8011085e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:45.823530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf58a1-75c1-451c-97ad-7da8011085e2 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:45.827970 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 726/1453] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:24:45.829089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10b1a4-902d-4832-b423-d6f10b9afea4 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:45.831889 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bd10b1a4-902d-4832-b423-d6f10b9afea4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:45.871869 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10b1a4-902d-4832-b423-d6f10b9afea4 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:45.879750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018762c-12c3-4207-a438-9cdafa3988f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:45.880350 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018762c-12c3-4207-a438-9cdafa3988f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:45.881158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018762c-12c3-4207-a438-9cdafa3988f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:45.881468 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018762c-12c3-4207-a438-9cdafa3988f4 admin admin] PATH_INFO: `/v3/projects/a8c9bec21f2c47b290ab18a4156152ff` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:45.883383 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018762c-12c3-4207-a438-9cdafa3988f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c9bec21f2c47b290ab18a4156152ff)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:45.887220 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018762c-12c3-4207-a438-9cdafa3988f4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:45.930393 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 728/1454] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:45 2022] POST /identity/v3/projects => generated 379 bytes in 697 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:45.932568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf58a1-75c1-451c-97ad-7da8011085e2 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:45.935849 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-16cf58a1-75c1-451c-97ad-7da8011085e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:46.007158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:46.067320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3018762c-12c3-4207-a438-9cdafa3988f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c9bec21f2c47b290ab18a4156152ff', 'request_id': 'req-3018762c-12c3-4207-a438-9cdafa3988f4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.094473 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3018762c-12c3-4207-a438-9cdafa3988f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c9bec21f2c47b290ab18a4156152ff', 'request_id': 'req-3018762c-12c3-4207-a438-9cdafa3988f4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.104569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3018762c-12c3-4207-a438-9cdafa3988f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c9bec21f2c47b290ab18a4156152ff', 'request_id': 'req-3018762c-12c3-4207-a438-9cdafa3988f4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.114097 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf58a1-75c1-451c-97ad-7da8011085e2 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:46.185202 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3018762c-12c3-4207-a438-9cdafa3988f4 admin admin] The token cache is being invalidate because project a8c9bec21f2c47b290ab18a4156152ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:46.185202 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3018762c-12c3-4207-a438-9cdafa3988f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.192723 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3018762c-12c3-4207-a438-9cdafa3988f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.203155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 727/1455] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:45 2022] DELETE /identity/v3/projects/a8c9bec21f2c47b290ab18a4156152ff => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:46.203155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3d5d9-371f-432b-a033-578c412382c3 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:46.203155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3d5d9-371f-432b-a033-578c412382c3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:46.203155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3d5d9-371f-432b-a033-578c412382c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:46.248214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-75c3d5d9-371f-432b-a033-578c412382c3 None None] MFA Rules not processed for user `6ff451e43cea4bb0a5fd73491a05bf82`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:46.249611 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-75c3d5d9-371f-432b-a033-578c412382c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:46.322362 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:46.322930 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:46.323233 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:46.323515 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] PATH_INFO: `/v3/projects/549de84684f646b0a4b237a0461ca160` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:46.325700 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=549de84684f646b0a4b237a0461ca160)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:46.327690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:46.382446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 728/1456] 10.222.0.25 () {58 vars in 1015 bytes} [Fri Oct 14 02:24:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:46.383541 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3d5d9-371f-432b-a033-578c412382c3 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:46.385293 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-75c3d5d9-371f-432b-a033-578c412382c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:46.413826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3d5d9-371f-432b-a033-578c412382c3 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:46.430372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c05d87-902c-458a-bd99-3dd39b8f5b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:46.431099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c05d87-902c-458a-bd99-3dd39b8f5b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:46.431995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c05d87-902c-458a-bd99-3dd39b8f5b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:46.431995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c05d87-902c-458a-bd99-3dd39b8f5b0d admin admin] PATH_INFO: `/v3/users/7241784ff34e40458e0d3f214aac32e5` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:46.432900 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c05d87-902c-458a-bd99-3dd39b8f5b0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7241784ff34e40458e0d3f214aac32e5)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:46.434867 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c05d87-902c-458a-bd99-3dd39b8f5b0d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:46.478918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '549de84684f646b0a4b237a0461ca160', 'request_id': 'req-a4739dbc-72f3-4277-bb84-d85308dc46f9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.479550 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:46.498346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '549de84684f646b0a4b237a0461ca160', 'request_id': 'req-a4739dbc-72f3-4277-bb84-d85308dc46f9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.517165 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '549de84684f646b0a4b237a0461ca160', 'request_id': 'req-a4739dbc-72f3-4277-bb84-d85308dc46f9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.530962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-19c05d87-902c-458a-bd99-3dd39b8f5b0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7241784ff34e40458e0d3f214aac32e5'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.541741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] The token cache is being invalidate because project 549de84684f646b0a4b237a0461ca160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:46.541977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.549184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-19c05d87-902c-458a-bd99-3dd39b8f5b0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7241784ff34e40458e0d3f214aac32e5'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.550162 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.558323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 729/1457] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:45 2022] DELETE /identity/v3/projects/549de84684f646b0a4b237a0461ca160 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:46.558323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:46.558640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-19c05d87-902c-458a-bd99-3dd39b8f5b0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7241784ff34e40458e0d3f214aac32e5'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.560359 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:46.591526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-19c05d87-902c-458a-bd99-3dd39b8f5b0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7241784ff34e40458e0d3f214aac32e5'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.611915 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-19c05d87-902c-458a-bd99-3dd39b8f5b0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7241784ff34e40458e0d3f214aac32e5'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:46.626069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 729/1458] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:46 2022] DELETE /identity/v3/users/7241784ff34e40458e0d3f214aac32e5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:46.639413 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593aabf-daf6-4aca-a9da-dd604164dc86 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:46.640237 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593aabf-daf6-4aca-a9da-dd604164dc86 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:46.640640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593aabf-daf6-4aca-a9da-dd604164dc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:46.728553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9593aabf-daf6-4aca-a9da-dd604164dc86 None None] MFA Rules not processed for user `e4ebab767c054b289e2279659a1cf2e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:46.728553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9593aabf-daf6-4aca-a9da-dd604164dc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:46.742646 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4739dbc-72f3-4277-bb84-d85308dc46f9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:46.795235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:46.975584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fe5a9-6475-4082-81f8-3ef59cdf0cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:46.979979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 730/1459] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:46.981068 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593aabf-daf6-4aca-a9da-dd604164dc86 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:46.982541 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9593aabf-daf6-4aca-a9da-dd604164dc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:46.997779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fe5a9-6475-4082-81f8-3ef59cdf0cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:47.004764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fe5a9-6475-4082-81f8-3ef59cdf0cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:47.005227 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593aabf-daf6-4aca-a9da-dd604164dc86 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:47.006431 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fe5a9-6475-4082-81f8-3ef59cdf0cc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:47.007264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fe5a9-6475-4082-81f8-3ef59cdf0cc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:47.013724 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c1016-6545-41c3-a817-46b882d0b222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:47.014386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c1016-6545-41c3-a817-46b882d0b222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:47.014682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c1016-6545-41c3-a817-46b882d0b222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:47.014987 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c1016-6545-41c3-a817-46b882d0b222 admin admin] PATH_INFO: `/v3/users/05777325449745f89eca267f9006c2a2` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:47.016287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c1016-6545-41c3-a817-46b882d0b222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05777325449745f89eca267f9006c2a2)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:47.018857 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c1016-6545-41c3-a817-46b882d0b222 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:47.026736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fe5a9-6475-4082-81f8-3ef59cdf0cc6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:47.116891 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2c2c1016-6545-41c3-a817-46b882d0b222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05777325449745f89eca267f9006c2a2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:47.131591 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2c2c1016-6545-41c3-a817-46b882d0b222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05777325449745f89eca267f9006c2a2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:47.132249 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2c2c1016-6545-41c3-a817-46b882d0b222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05777325449745f89eca267f9006c2a2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:47.163412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 730/1460] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:46 2022] POST /identity/v3/users => generated 296 bytes in 605 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:47.164456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fe5a9-6475-4082-81f8-3ef59cdf0cc6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:47.169160 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2c2c1016-6545-41c3-a817-46b882d0b222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05777325449745f89eca267f9006c2a2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:47.174313 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-710fe5a9-6475-4082-81f8-3ef59cdf0cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:47.200120 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2c2c1016-6545-41c3-a817-46b882d0b222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05777325449745f89eca267f9006c2a2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:47.216818 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 731/1461] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:46 2022] DELETE /identity/v3/users/05777325449745f89eca267f9006c2a2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:47.218072 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd61fbe-acb9-4890-b987-05fe078d0f2e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:47.218386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd61fbe-acb9-4890-b987-05fe078d0f2e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:47.218670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd61fbe-acb9-4890-b987-05fe078d0f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:47.285381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-cbd61fbe-acb9-4890-b987-05fe078d0f2e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:47.288275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cbd61fbe-acb9-4890-b987-05fe078d0f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:47.332658 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:47.502904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fe5a9-6475-4082-81f8-3ef59cdf0cc6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:47.528787 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 732/1462] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:47.538286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c54d07-3bd2-4ee1-9a5d-2442cc6020e6 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:47.538973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c54d07-3bd2-4ee1-9a5d-2442cc6020e6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:47.539475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c54d07-3bd2-4ee1-9a5d-2442cc6020e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:47.594325 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-40c54d07-3bd2-4ee1-9a5d-2442cc6020e6 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:47.599419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-40c54d07-3bd2-4ee1-9a5d-2442cc6020e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:47.630492 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 733/1463] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:47.635156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c54d07-3bd2-4ee1-9a5d-2442cc6020e6 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:47.638511 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-40c54d07-3bd2-4ee1-9a5d-2442cc6020e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:47.695707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c54d07-3bd2-4ee1-9a5d-2442cc6020e6 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:47.713543 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b712c36-832e-48b7-9a24-e328b10a03aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:47.715712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b712c36-832e-48b7-9a24-e328b10a03aa admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:47.716551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b712c36-832e-48b7-9a24-e328b10a03aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:47.716957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b712c36-832e-48b7-9a24-e328b10a03aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:47.719692 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b712c36-832e-48b7-9a24-e328b10a03aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:47.725442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b712c36-832e-48b7-9a24-e328b10a03aa admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:47.775477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa9190-9cfd-467a-842d-26efe1aace95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:47.776701 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 734/1464] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:47.777181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa9190-9cfd-467a-842d-26efe1aace95 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:47.777730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa9190-9cfd-467a-842d-26efe1aace95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:47.778372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa9190-9cfd-467a-842d-26efe1aace95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:47.782902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7afa9190-9cfd-467a-842d-26efe1aace95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:47.787506 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ad700-b0cb-47bf-873f-be107d71ad6a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:47.787809 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ad700-b0cb-47bf-873f-be107d71ad6a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:47.790471 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ad700-b0cb-47bf-873f-be107d71ad6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:47.813962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa9190-9cfd-467a-842d-26efe1aace95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:47.834818 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa9190-9cfd-467a-842d-26efe1aace95 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:47.874472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 731/1465] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:47.878015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-aa4ad700-b0cb-47bf-873f-be107d71ad6a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:47.883012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4f071-1a90-4e4d-a8bf-9d7847c02bbe None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:47.883461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4f071-1a90-4e4d-a8bf-9d7847c02bbe None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:47.884374 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4f071-1a90-4e4d-a8bf-9d7847c02bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:47.885250 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-aa4ad700-b0cb-47bf-873f-be107d71ad6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:47.924093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 735/1466] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:47.925244 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ad700-b0cb-47bf-873f-be107d71ad6a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:47.934290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-aa4ad700-b0cb-47bf-873f-be107d71ad6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:47.957831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-aaa4f071-1a90-4e4d-a8bf-9d7847c02bbe None None] MFA Rules not processed for user `79365a5e4ae2431d91a4aa603ba58fee`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:47.965585 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-aaa4f071-1a90-4e4d-a8bf-9d7847c02bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:47.982046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ad700-b0cb-47bf-873f-be107d71ad6a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:47.999602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4d28d-3b25-414c-b532-296286a3914a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:48.000246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4d28d-3b25-414c-b532-296286a3914a service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:48.000532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4d28d-3b25-414c-b532-296286a3914a service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:48.000809 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4d28d-3b25-414c-b532-296286a3914a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:48.038723 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4d28d-3b25-414c-b532-296286a3914a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:48.042106 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4d28d-3b25-414c-b532-296286a3914a service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:48.106429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 736/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:48.106429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4d28d-3b25-414c-b532-296286a3914a service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:48.145199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4d28d-3b25-414c-b532-296286a3914a service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:48.177204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13125341-780b-46b9-9f98-6f09a6a323de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:48.201603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13125341-780b-46b9-9f98-6f09a6a323de admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:48.201912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13125341-780b-46b9-9f98-6f09a6a323de admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:48.204937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 732/1468] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:48.205929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4f071-1a90-4e4d-a8bf-9d7847c02bbe None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:48.206052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13125341-780b-46b9-9f98-6f09a6a323de admin admin] PATH_INFO: `/v3/projects/2cebd13aeeed4462b4a6eb2f76c82e4f/users/5b98c9dce6be48129aadc64bbfae57af/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:48.254909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4f071-1a90-4e4d-a8bf-9d7847c02bbe None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:48.270089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13125341-780b-46b9-9f98-6f09a6a323de admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cebd13aeeed4462b4a6eb2f76c82e4f, user_id=5b98c9dce6be48129aadc64bbfae57af, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:48.280106 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa54d81-dd2d-43b8-8461-4425a1e305ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:48.280833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa54d81-dd2d-43b8-8461-4425a1e305ad admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:48.281083 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa54d81-dd2d-43b8-8461-4425a1e305ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:48.281363 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa54d81-dd2d-43b8-8461-4425a1e305ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:48.282017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa54d81-dd2d-43b8-8461-4425a1e305ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:48.282380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa54d81-dd2d-43b8-8461-4425a1e305ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:48.282488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13125341-780b-46b9-9f98-6f09a6a323de admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:48.284119 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa54d81-dd2d-43b8-8461-4425a1e305ad admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:48.293645 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:48.323788 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 733/1469] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:48.328116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa54d81-dd2d-43b8-8461-4425a1e305ad admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:48.336526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 737/1470] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:48 2022] PUT /identity/v3/projects/2cebd13aeeed4462b4a6eb2f76c82e4f/users/5b98c9dce6be48129aadc64bbfae57af/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:48.340784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13125341-780b-46b9-9f98-6f09a6a323de admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:48.357049 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa54d81-dd2d-43b8-8461-4425a1e305ad admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:48.529864 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9be16-7c9f-4948-96bf-4aa5f7c70f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:48.530556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9be16-7c9f-4948-96bf-4aa5f7c70f08 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:48.530779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9be16-7c9f-4948-96bf-4aa5f7c70f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:48.531055 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9be16-7c9f-4948-96bf-4aa5f7c70f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:48.531728 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9be16-7c9f-4948-96bf-4aa5f7c70f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:48.533682 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9be16-7c9f-4948-96bf-4aa5f7c70f08 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:48.642527 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 734/1471] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:48 2022] POST /identity/v3/projects => generated 377 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:24:48.651204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13125341-780b-46b9-9f98-6f09a6a323de admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:48.652281 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9be16-7c9f-4948-96bf-4aa5f7c70f08 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:48.834955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a2d35-9d7a-48bc-9218-604b3e237df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:48.838031 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2d35-9d7a-48bc-9218-604b3e237df2 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:48.838557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2d35-9d7a-48bc-9218-604b3e237df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:48.839350 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2d35-9d7a-48bc-9218-604b3e237df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:48.840927 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a2d35-9d7a-48bc-9218-604b3e237df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:48.843499 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a2d35-9d7a-48bc-9218-604b3e237df2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:48.845930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:48.861928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 738/1472] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 519 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:48.866333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a2d35-9d7a-48bc-9218-604b3e237df2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:48.891531 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a2d35-9d7a-48bc-9218-604b3e237df2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:48.915880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178620b5-9d1b-496d-94a6-bcb5f0f54065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:48.917014 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178620b5-9d1b-496d-94a6-bcb5f0f54065 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:48.917859 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178620b5-9d1b-496d-94a6-bcb5f0f54065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:48.918185 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178620b5-9d1b-496d-94a6-bcb5f0f54065 admin admin] PATH_INFO: `/v3/projects/2cebd13aeeed4462b4a6eb2f76c82e4f/users/5b98c9dce6be48129aadc64bbfae57af/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:48.921366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178620b5-9d1b-496d-94a6-bcb5f0f54065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cebd13aeeed4462b4a6eb2f76c82e4f, user_id=5b98c9dce6be48129aadc64bbfae57af, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:48.926837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178620b5-9d1b-496d-94a6-bcb5f0f54065 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:48.928054 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9be16-7c9f-4948-96bf-4aa5f7c70f08 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:48.947361 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 739/1473] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:48 2022] PUT /identity/v3/projects/2cebd13aeeed4462b4a6eb2f76c82e4f/users/5b98c9dce6be48129aadc64bbfae57af/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:48.962249 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343de0b-e2cb-48c3-af6e-9f3367fd8d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:48.963054 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343de0b-e2cb-48c3-af6e-9f3367fd8d27 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:48.963374 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343de0b-e2cb-48c3-af6e-9f3367fd8d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:48.963647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343de0b-e2cb-48c3-af6e-9f3367fd8d27 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:48.964317 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343de0b-e2cb-48c3-af6e-9f3367fd8d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:48.965004 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d2058-c64b-43dc-8005-9b962cd6e516 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:48.965362 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d2058-c64b-43dc-8005-9b962cd6e516 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:48.966009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d2058-c64b-43dc-8005-9b962cd6e516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:48.968615 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343de0b-e2cb-48c3-af6e-9f3367fd8d27 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:49.029109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-be4d2058-c64b-43dc-8005-9b962cd6e516 None None] MFA Rules not processed for user `5b98c9dce6be48129aadc64bbfae57af`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:49.033351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-be4d2058-c64b-43dc-8005-9b962cd6e516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:49.052078 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 735/1474] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:48 2022] POST /identity/v3/users => generated 289 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:49.053222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343de0b-e2cb-48c3-af6e-9f3367fd8d27 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:49.084244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343de0b-e2cb-48c3-af6e-9f3367fd8d27 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:49.322786 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 740/1475] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:49.323613 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d2058-c64b-43dc-8005-9b962cd6e516 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:49.325427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-be4d2058-c64b-43dc-8005-9b962cd6e516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:49.335805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd25456-c722-4888-a66c-48e64ba1d64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:49.336402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd25456-c722-4888-a66c-48e64ba1d64c admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:49.336644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd25456-c722-4888-a66c-48e64ba1d64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:49.336911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd25456-c722-4888-a66c-48e64ba1d64c admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:49.337690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd25456-c722-4888-a66c-48e64ba1d64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:49.339651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd25456-c722-4888-a66c-48e64ba1d64c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:49.366497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d2058-c64b-43dc-8005-9b962cd6e516 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:49.372492 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:49.373156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:49.373439 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:49.373769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] PATH_INFO: `/v3/projects/1f408bbfbd60422e863605b5fbe1878f` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:49.376920 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f408bbfbd60422e863605b5fbe1878f)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:49.381373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 736/1476] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:49.382388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd25456-c722-4888-a66c-48e64ba1d64c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:49.388408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3dd25456-c722-4888-a66c-48e64ba1d64c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:49.390407 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:49.415543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:49.511428 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f408bbfbd60422e863605b5fbe1878f', 'request_id': 'req-71e7ecd5-a261-4528-b8f8-e53bc9067b57'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:49.525600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f408bbfbd60422e863605b5fbe1878f', 'request_id': 'req-71e7ecd5-a261-4528-b8f8-e53bc9067b57'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:49.563320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f408bbfbd60422e863605b5fbe1878f', 'request_id': 'req-71e7ecd5-a261-4528-b8f8-e53bc9067b57'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:49.574896 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd25456-c722-4888-a66c-48e64ba1d64c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:49.601662 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] The token cache is being invalidate because project 1f408bbfbd60422e863605b5fbe1878f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:49.618025 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:49.621004 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:49.631159 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 741/1477] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:49 2022] DELETE /identity/v3/projects/1f408bbfbd60422e863605b5fbe1878f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:49.635385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:49.639286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:49.682087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0075366-1da1-4908-8726-04cb235ab844 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:49.683221 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0075366-1da1-4908-8726-04cb235ab844 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:49.684396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0075366-1da1-4908-8726-04cb235ab844 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:49.684959 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0075366-1da1-4908-8726-04cb235ab844 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:49.692651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e0075366-1da1-4908-8726-04cb235ab844 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:49.764526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:49.804021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7ecd5-a261-4528-b8f8-e53bc9067b57 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:49.820023 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292402b7-f9b5-44ca-964d-15c040db7aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:49.821099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292402b7-f9b5-44ca-964d-15c040db7aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:49.822090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292402b7-f9b5-44ca-964d-15c040db7aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:49.822583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292402b7-f9b5-44ca-964d-15c040db7aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:49.823605 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292402b7-f9b5-44ca-964d-15c040db7aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:49.828928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292402b7-f9b5-44ca-964d-15c040db7aa9 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:49.838245 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0075366-1da1-4908-8726-04cb235ab844 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:49.841776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0075366-1da1-4908-8726-04cb235ab844 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:49.857619 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 737/1479] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:49.859346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342a042-1b60-471a-a52a-46420bc57f26 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:49.859830 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342a042-1b60-471a-a52a-46420bc57f26 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:49.860558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342a042-1b60-471a-a52a-46420bc57f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:49.867904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 742/1479] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:49.872518 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292402b7-f9b5-44ca-964d-15c040db7aa9 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:49.900929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b342a042-1b60-471a-a52a-46420bc57f26 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:49.903606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b342a042-1b60-471a-a52a-46420bc57f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:49.916719 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292402b7-f9b5-44ca-964d-15c040db7aa9 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:49.928861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 738/1480] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:49.953402 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bee391-22a7-4dad-bccd-f9da657fefc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:49.955336 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bee391-22a7-4dad-bccd-f9da657fefc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:49.955621 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bee391-22a7-4dad-bccd-f9da657fefc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:49.955899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bee391-22a7-4dad-bccd-f9da657fefc4 admin admin] PATH_INFO: `/v3/projects/35b239350d5240bfbe9805986151d082/users/3ad09cb4dbd34155bd9ce937c58fc7b5/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:49.957056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289b4c9-9eaf-4180-90d6-5ea6558b7f79 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:49.960709 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289b4c9-9eaf-4180-90d6-5ea6558b7f79 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:49.961002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289b4c9-9eaf-4180-90d6-5ea6558b7f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:49.995299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bee391-22a7-4dad-bccd-f9da657fefc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35b239350d5240bfbe9805986151d082, user_id=3ad09cb4dbd34155bd9ce937c58fc7b5, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:49.996804 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-7289b4c9-9eaf-4180-90d6-5ea6558b7f79 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:49.998211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7289b4c9-9eaf-4180-90d6-5ea6558b7f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:50.002844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bee391-22a7-4dad-bccd-f9da657fefc4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:50.010077 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 739/1481] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:50.014255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289b4c9-9eaf-4180-90d6-5ea6558b7f79 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:50.030955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 743/1482] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:49 2022] PUT /identity/v3/projects/35b239350d5240bfbe9805986151d082/users/3ad09cb4dbd34155bd9ce937c58fc7b5/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:50.038081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289b4c9-9eaf-4180-90d6-5ea6558b7f79 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:50.044163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b1f1e-2db9-4631-8365-a31acbc5837a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:50.044460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b1f1e-2db9-4631-8365-a31acbc5837a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:50.047634 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b1f1e-2db9-4631-8365-a31acbc5837a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:50.118204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-564b1f1e-2db9-4631-8365-a31acbc5837a None None] MFA Rules not processed for user `3ad09cb4dbd34155bd9ce937c58fc7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:50.119809 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-564b1f1e-2db9-4631-8365-a31acbc5837a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:50.241289 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c42ed7-c789-4408-8404-116169141e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:50.242016 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c42ed7-c789-4408-8404-116169141e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:50.242372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c42ed7-c789-4408-8404-116169141e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:50.242843 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c42ed7-c789-4408-8404-116169141e76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:50.244420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c42ed7-c789-4408-8404-116169141e76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:50.244877 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c42ed7-c789-4408-8404-116169141e76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:50.247116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c42ed7-c789-4408-8404-116169141e76 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:50.249686 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:50.257750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 744/1483] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:50.260397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b1f1e-2db9-4631-8365-a31acbc5837a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:50.263497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-564b1f1e-2db9-4631-8365-a31acbc5837a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:50.271116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 740/1484] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:50.271116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c42ed7-c789-4408-8404-116169141e76 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:50.289889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c42ed7-c789-4408-8404-116169141e76 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:50.296003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82717-a509-41d3-878a-b10953894b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:50.297063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82717-a509-41d3-878a-b10953894b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:50.297894 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82717-a509-41d3-878a-b10953894b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:50.298018 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82717-a509-41d3-878a-b10953894b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:50.299129 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec82717-a509-41d3-878a-b10953894b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:50.305387 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec82717-a509-41d3-878a-b10953894b7e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:50.367230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 741/1485] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:50 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:50.367230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82717-a509-41d3-878a-b10953894b7e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:50.476345 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b1f1e-2db9-4631-8365-a31acbc5837a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:50.556321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82717-a509-41d3-878a-b10953894b7e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:50.570467 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562b50d-b117-42e9-875d-30a5d46eeade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:50.571148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562b50d-b117-42e9-875d-30a5d46eeade admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:50.571426 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562b50d-b117-42e9-875d-30a5d46eeade admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:50.571704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562b50d-b117-42e9-875d-30a5d46eeade admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:50.572354 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562b50d-b117-42e9-875d-30a5d46eeade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:50.582969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562b50d-b117-42e9-875d-30a5d46eeade admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:50.647995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28215a9-6b8a-4f2c-8f40-6c149f3f3dff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:50.648953 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28215a9-6b8a-4f2c-8f40-6c149f3f3dff service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:50.649445 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28215a9-6b8a-4f2c-8f40-6c149f3f3dff service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:50.649941 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28215a9-6b8a-4f2c-8f40-6c149f3f3dff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:50.652213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e28215a9-6b8a-4f2c-8f40-6c149f3f3dff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:50.653037 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 742/1486] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:50 2022] POST /identity/v3/projects => generated 387 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:50.653907 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562b50d-b117-42e9-875d-30a5d46eeade admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:50.701084 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:50.898743 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28215a9-6b8a-4f2c-8f40-6c149f3f3dff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:50.906387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28215a9-6b8a-4f2c-8f40-6c149f3f3dff service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:50.929663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562b50d-b117-42e9-875d-30a5d46eeade admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:50.944247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 745/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:50.944987 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28215a9-6b8a-4f2c-8f40-6c149f3f3dff service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:50.947224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e4149-586b-4590-b23b-172e0b94bfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:50.947994 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e4149-586b-4590-b23b-172e0b94bfe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:50.948309 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e4149-586b-4590-b23b-172e0b94bfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:50.948584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e4149-586b-4590-b23b-172e0b94bfe5 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:50.949448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e4149-586b-4590-b23b-172e0b94bfe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:50.951648 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e4149-586b-4590-b23b-172e0b94bfe5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:50.988338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28215a9-6b8a-4f2c-8f40-6c149f3f3dff service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:51.000860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e20353-9aa5-4183-9d44-c100adcfe54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:51.004356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e20353-9aa5-4183-9d44-c100adcfe54b admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:51.004671 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e20353-9aa5-4183-9d44-c100adcfe54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:51.005435 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e20353-9aa5-4183-9d44-c100adcfe54b admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:51.006229 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e20353-9aa5-4183-9d44-c100adcfe54b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:51.015593 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e20353-9aa5-4183-9d44-c100adcfe54b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:51.038293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 743/1488] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:50 2022] POST /identity/v3/users => generated 295 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:51.038293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c979a7-1650-4490-9c5b-123726c6c4ac None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:51.038293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c979a7-1650-4490-9c5b-123726c6c4ac None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:51.038293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c979a7-1650-4490-9c5b-123726c6c4ac None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:51.044453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 744/1489] 10.222.0.18 () {54 vars in 921 bytes} [Fri Oct 14 02:24:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:24:51.048561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c979a7-1650-4490-9c5b-123726c6c4ac None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:51.086069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 746/1490] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:50 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:51.091406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e20353-9aa5-4183-9d44-c100adcfe54b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:51.105309 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c979a7-1650-4490-9c5b-123726c6c4ac None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:51.118360 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e20353-9aa5-4183-9d44-c100adcfe54b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:51.123578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825568ba-1e12-4676-8d27-467b9c86237e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:51.125660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825568ba-1e12-4676-8d27-467b9c86237e admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:51.125963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825568ba-1e12-4676-8d27-467b9c86237e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:51.126241 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825568ba-1e12-4676-8d27-467b9c86237e admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:51.126907 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825568ba-1e12-4676-8d27-467b9c86237e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:51.128583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825568ba-1e12-4676-8d27-467b9c86237e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:51.130544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537006b-b4f3-4607-87e0-495d7fee51d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:51.131254 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537006b-b4f3-4607-87e0-495d7fee51d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:51.131540 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537006b-b4f3-4607-87e0-495d7fee51d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:51.132140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537006b-b4f3-4607-87e0-495d7fee51d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:51.132670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537006b-b4f3-4607-87e0-495d7fee51d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:51.137410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537006b-b4f3-4607-87e0-495d7fee51d8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:51.170166 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 747/1491] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:51.171187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537006b-b4f3-4607-87e0-495d7fee51d8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:51.185705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 745/1492] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:51.188242 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825568ba-1e12-4676-8d27-467b9c86237e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:51.195026 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537006b-b4f3-4607-87e0-495d7fee51d8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:51.210344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28d5e3-5d1e-46b5-bcc9-4f08e85a6e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:51.211375 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28d5e3-5d1e-46b5-bcc9-4f08e85a6e7d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:51.211853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28d5e3-5d1e-46b5-bcc9-4f08e85a6e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:51.214213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825568ba-1e12-4676-8d27-467b9c86237e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:51.216318 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28d5e3-5d1e-46b5-bcc9-4f08e85a6e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:51.217293 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28d5e3-5d1e-46b5-bcc9-4f08e85a6e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:51.219943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28d5e3-5d1e-46b5-bcc9-4f08e85a6e7d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:51.232404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdd6ec-3a60-40ec-8bea-4bbb80128eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:51.233123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdd6ec-3a60-40ec-8bea-4bbb80128eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:51.233429 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdd6ec-3a60-40ec-8bea-4bbb80128eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:51.233712 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdd6ec-3a60-40ec-8bea-4bbb80128eaa admin admin] PATH_INFO: `/v3/projects/b7da6346a3734f72914c52dad142e918/users/83790aa5a2914dd6ab86f5f3944e7ca3/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:51.237432 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 748/1493] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:51.241233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28d5e3-5d1e-46b5-bcc9-4f08e85a6e7d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:51.253829 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28d5e3-5d1e-46b5-bcc9-4f08e85a6e7d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:51.271516 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdd6ec-3a60-40ec-8bea-4bbb80128eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7da6346a3734f72914c52dad142e918, user_id=83790aa5a2914dd6ab86f5f3944e7ca3, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:51.277866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdd6ec-3a60-40ec-8bea-4bbb80128eaa admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:51.282008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30733ad3-3ba9-4ccd-8ee0-c9d59a1c6622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:51.283284 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733ad3-3ba9-4ccd-8ee0-c9d59a1c6622 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:51.283931 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733ad3-3ba9-4ccd-8ee0-c9d59a1c6622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:51.284527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733ad3-3ba9-4ccd-8ee0-c9d59a1c6622 admin admin] PATH_INFO: `/v3/projects/e84585f806fe405b9981a3c1f6e91263/users/86f1ab1b9e634a91a5884729f1d30a87/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:51.330234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30733ad3-3ba9-4ccd-8ee0-c9d59a1c6622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e84585f806fe405b9981a3c1f6e91263, user_id=86f1ab1b9e634a91a5884729f1d30a87, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:51.333794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 746/1494] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:51 2022] PUT /identity/v3/projects/b7da6346a3734f72914c52dad142e918/users/83790aa5a2914dd6ab86f5f3944e7ca3/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:51.334354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30733ad3-3ba9-4ccd-8ee0-c9d59a1c6622 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:51.339340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdd6ec-3a60-40ec-8bea-4bbb80128eaa admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:51.373499 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 749/1495] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:51 2022] PUT /identity/v3/projects/e84585f806fe405b9981a3c1f6e91263/users/86f1ab1b9e634a91a5884729f1d30a87/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:51.381774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb350a0-2b31-4b66-b9e1-d8b8ffe66b95 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:51.382208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb350a0-2b31-4b66-b9e1-d8b8ffe66b95 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:51.382473 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb350a0-2b31-4b66-b9e1-d8b8ffe66b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:51.388796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:51.458201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ecb350a0-2b31-4b66-b9e1-d8b8ffe66b95 None None] MFA Rules not processed for user `de13be55b4034477baf4c029c1a79757`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:51.461387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ecb350a0-2b31-4b66-b9e1-d8b8ffe66b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:51.555334 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdd6ec-3a60-40ec-8bea-4bbb80128eaa admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:51.726899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 750/1496] 10.222.0.18 () {58 vars in 1018 bytes} [Fri Oct 14 02:24:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:51.728443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c21d6-6a69-4712-bbb6-7801ef59f60a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:51.728544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c21d6-6a69-4712-bbb6-7801ef59f60a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:51.728892 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c21d6-6a69-4712-bbb6-7801ef59f60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:51.799406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d452d71-f382-426c-a01f-dc1bc423b929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:51.800080 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d452d71-f382-426c-a01f-dc1bc423b929 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:51.801956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d452d71-f382-426c-a01f-dc1bc423b929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:51.801956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d452d71-f382-426c-a01f-dc1bc423b929 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:51.801956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d452d71-f382-426c-a01f-dc1bc423b929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:51.803376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d452d71-f382-426c-a01f-dc1bc423b929 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:51.832676 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-530c21d6-6a69-4712-bbb6-7801ef59f60a None None] MFA Rules not processed for user `86f1ab1b9e634a91a5884729f1d30a87`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:51.834373 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-530c21d6-6a69-4712-bbb6-7801ef59f60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:51.842974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 747/1497] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 505 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:51.843958 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d452d71-f382-426c-a01f-dc1bc423b929 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:51.866135 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d452d71-f382-426c-a01f-dc1bc423b929 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:51.884226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077b380-bc73-49d4-8d6d-b0f7363c974a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:51.884925 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077b380-bc73-49d4-8d6d-b0f7363c974a admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:51.885204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077b380-bc73-49d4-8d6d-b0f7363c974a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:51.885481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077b380-bc73-49d4-8d6d-b0f7363c974a admin admin] PATH_INFO: `/v3/projects/b7da6346a3734f72914c52dad142e918/users/83790aa5a2914dd6ab86f5f3944e7ca3/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:51.890249 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077b380-bc73-49d4-8d6d-b0f7363c974a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7da6346a3734f72914c52dad142e918, user_id=83790aa5a2914dd6ab86f5f3944e7ca3, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:51.900227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077b380-bc73-49d4-8d6d-b0f7363c974a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:51.959581 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 748/1498] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:51 2022] PUT /identity/v3/projects/b7da6346a3734f72914c52dad142e918/users/83790aa5a2914dd6ab86f5f3944e7ca3/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:51.962446 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c70c8d-e9b6-4c8b-91f3-356e1bbdb3a9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:51.962743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c70c8d-e9b6-4c8b-91f3-356e1bbdb3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:51.964167 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c70c8d-e9b6-4c8b-91f3-356e1bbdb3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:52.047830 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a4c70c8d-e9b6-4c8b-91f3-356e1bbdb3a9 None None] MFA Rules not processed for user `83790aa5a2914dd6ab86f5f3944e7ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:52.049600 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a4c70c8d-e9b6-4c8b-91f3-356e1bbdb3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:52.252509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:52.321822 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 749/1499] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:52.329979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c70c8d-e9b6-4c8b-91f3-356e1bbdb3a9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:52.342360 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 751/1500] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 612 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:52.345104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c21d6-6a69-4712-bbb6-7801ef59f60a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:52.385829 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c21d6-6a69-4712-bbb6-7801ef59f60a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:52.597366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:52.643755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293e797-4315-4723-9f2a-1c764bb1e922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:52.645846 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293e797-4315-4723-9f2a-1c764bb1e922 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:52.646298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293e797-4315-4723-9f2a-1c764bb1e922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:52.647516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293e797-4315-4723-9f2a-1c764bb1e922 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:52.648949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293e797-4315-4723-9f2a-1c764bb1e922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:52.651449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293e797-4315-4723-9f2a-1c764bb1e922 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:52.704277 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c70c8d-e9b6-4c8b-91f3-356e1bbdb3a9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:52.742509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa4450-0892-4389-a044-764546f47151 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:52.742509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4450-0892-4389-a044-764546f47151 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:52.742509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4450-0892-4389-a044-764546f47151 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:52.742509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4450-0892-4389-a044-764546f47151 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:52.819168 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 752/1501] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:52 2022] POST /identity/v3/projects => generated 383 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:24:52.819168 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293e797-4315-4723-9f2a-1c764bb1e922 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:52.827378 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4293e797-4315-4723-9f2a-1c764bb1e922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:52.999952 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa4450-0892-4389-a044-764546f47151 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:53.004163 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa4450-0892-4389-a044-764546f47151 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:53.038665 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 750/1502] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 708 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:53.040270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa4450-0892-4389-a044-764546f47151 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:53.088191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa4450-0892-4389-a044-764546f47151 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:53.117068 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293e797-4315-4723-9f2a-1c764bb1e922 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:53.123023 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f908b-74eb-469d-b10b-7b02c86a493b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:53.127945 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f908b-74eb-469d-b10b-7b02c86a493b admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:53.129872 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f908b-74eb-469d-b10b-7b02c86a493b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:53.130185 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f908b-74eb-469d-b10b-7b02c86a493b admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:53.131270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f908b-74eb-469d-b10b-7b02c86a493b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:53.137061 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:53.137947 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:53.138446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:53.139113 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] PATH_INFO: `/v3/projects/dcbc1920380a46b0a66704f886bf9c18` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:53.142353 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcbc1920380a46b0a66704f886bf9c18)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:53.143444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f908b-74eb-469d-b10b-7b02c86a493b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:53.150517 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:53.237647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 751/1503] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:53 2022] POST /identity/v3/users => generated 298 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:53.247385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f908b-74eb-469d-b10b-7b02c86a493b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:53.254482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-790f908b-74eb-469d-b10b-7b02c86a493b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:53.359660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcbc1920380a46b0a66704f886bf9c18', 'request_id': 'req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:53.393079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcbc1920380a46b0a66704f886bf9c18', 'request_id': 'req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:53.435391 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcbc1920380a46b0a66704f886bf9c18', 'request_id': 'req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:53.455102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:53.528440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] The token cache is being invalidate because project dcbc1920380a46b0a66704f886bf9c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:53.529998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:53.533615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:53.543285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 753/1504] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:52 2022] DELETE /identity/v3/projects/dcbc1920380a46b0a66704f886bf9c18 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:53.545802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:53.554889 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:53.721721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f908b-74eb-469d-b10b-7b02c86a493b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:53.956753 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b304f-5eeb-4f50-a08d-eabc5f4480ff admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:53.976729 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46ca4e-0543-449b-84f5-afad8fe34690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:53.977512 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46ca4e-0543-449b-84f5-afad8fe34690 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:53.978233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46ca4e-0543-449b-84f5-afad8fe34690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:53.978805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46ca4e-0543-449b-84f5-afad8fe34690 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:53.980865 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46ca4e-0543-449b-84f5-afad8fe34690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:53.983668 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46ca4e-0543-449b-84f5-afad8fe34690 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:53.983790 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:54.023860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85444bdc-900d-4375-8852-1a0ecb90a84c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:54.024551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85444bdc-900d-4375-8852-1a0ecb90a84c service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:54.027765 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85444bdc-900d-4375-8852-1a0ecb90a84c service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:54.031447 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85444bdc-900d-4375-8852-1a0ecb90a84c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:54.036872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 754/1505] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 489 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:54.036872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46ca4e-0543-449b-84f5-afad8fe34690 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:54.043284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-85444bdc-900d-4375-8852-1a0ecb90a84c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:54.054911 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0c46ca4e-0543-449b-84f5-afad8fe34690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:54.111713 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46ca4e-0543-449b-84f5-afad8fe34690 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:54.118899 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85444bdc-900d-4375-8852-1a0ecb90a84c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:54.124199 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85444bdc-900d-4375-8852-1a0ecb90a84c service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:54.138939 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26290e-979c-47df-89b7-3db973adc476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:54.153201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26290e-979c-47df-89b7-3db973adc476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:54.153676 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26290e-979c-47df-89b7-3db973adc476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:54.156702 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26290e-979c-47df-89b7-3db973adc476 admin admin] PATH_INFO: `/v3/users/0c464e2a20514b55a92936bad246e29b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:54.168016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26290e-979c-47df-89b7-3db973adc476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c464e2a20514b55a92936bad246e29b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:54.170192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26290e-979c-47df-89b7-3db973adc476 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:54.202489 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 752/1506] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 964 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:54.209995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85444bdc-900d-4375-8852-1a0ecb90a84c service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:54.268764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85444bdc-900d-4375-8852-1a0ecb90a84c service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:54.285868 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ec103-9e37-4259-9dfc-d69b85f8bcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:54.287479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ec103-9e37-4259-9dfc-d69b85f8bcd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:54.288167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3b26290e-979c-47df-89b7-3db973adc476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c464e2a20514b55a92936bad246e29b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:54.289832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ec103-9e37-4259-9dfc-d69b85f8bcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:54.290369 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ec103-9e37-4259-9dfc-d69b85f8bcd2 admin admin] PATH_INFO: `/v3/projects/8d24ef2a62564a749134cc89e67a6d37/users/b2d10151fb1849a8bc602b7a70bcc891/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:54.309873 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3b26290e-979c-47df-89b7-3db973adc476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c464e2a20514b55a92936bad246e29b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:54.310780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3b26290e-979c-47df-89b7-3db973adc476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c464e2a20514b55a92936bad246e29b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:54.351592 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ec103-9e37-4259-9dfc-d69b85f8bcd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d24ef2a62564a749134cc89e67a6d37, user_id=b2d10151fb1849a8bc602b7a70bcc891, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:54.352364 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3b26290e-979c-47df-89b7-3db973adc476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c464e2a20514b55a92936bad246e29b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:54.355903 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ec103-9e37-4259-9dfc-d69b85f8bcd2 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:54.378008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3b26290e-979c-47df-89b7-3db973adc476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c464e2a20514b55a92936bad246e29b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:54.389262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 753/1507] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:54 2022] PUT /identity/v3/projects/8d24ef2a62564a749134cc89e67a6d37/users/b2d10151fb1849a8bc602b7a70bcc891/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:54.390778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 755/1508] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:54 2022] DELETE /identity/v3/users/0c464e2a20514b55a92936bad246e29b => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:54.394627 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ec103-9e37-4259-9dfc-d69b85f8bcd2 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:54.398239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1f8ec103-9e37-4259-9dfc-d69b85f8bcd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:54.610570 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ec103-9e37-4259-9dfc-d69b85f8bcd2 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:54.794369 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba25723-3135-44c3-9de2-28a01fd42d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:54.795385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba25723-3135-44c3-9de2-28a01fd42d1e admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:54.795834 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba25723-3135-44c3-9de2-28a01fd42d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:54.797237 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba25723-3135-44c3-9de2-28a01fd42d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:54.798610 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba25723-3135-44c3-9de2-28a01fd42d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:54.807087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba25723-3135-44c3-9de2-28a01fd42d1e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:54.834993 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 754/1509] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:54.840765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26290e-979c-47df-89b7-3db973adc476 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:54.869056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26290e-979c-47df-89b7-3db973adc476 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:54.879843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a0eb7-756c-46ac-bd6b-aea18738c688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:54.881413 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a0eb7-756c-46ac-bd6b-aea18738c688 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:54.883287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a0eb7-756c-46ac-bd6b-aea18738c688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:54.884239 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a0eb7-756c-46ac-bd6b-aea18738c688 admin admin] PATH_INFO: `/v3/projects/8d24ef2a62564a749134cc89e67a6d37/users/b2d10151fb1849a8bc602b7a70bcc891/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:54.888922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a0eb7-756c-46ac-bd6b-aea18738c688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d24ef2a62564a749134cc89e67a6d37, user_id=b2d10151fb1849a8bc602b7a70bcc891, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:54.892156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a0eb7-756c-46ac-bd6b-aea18738c688 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:54.914185 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 756/1510] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:54 2022] PUT /identity/v3/projects/8d24ef2a62564a749134cc89e67a6d37/users/b2d10151fb1849a8bc602b7a70bcc891/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:54.926524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c1198-b025-48e9-8e1e-ce367cfdc88f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:54.926980 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c1198-b025-48e9-8e1e-ce367cfdc88f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:54.927583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c1198-b025-48e9-8e1e-ce367cfdc88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:55.074110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e84c1198-b025-48e9-8e1e-ce367cfdc88f None None] MFA Rules not processed for user `b2d10151fb1849a8bc602b7a70bcc891`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:55.078193 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e84c1198-b025-48e9-8e1e-ce367cfdc88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:55.292422 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:55.345009 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 755/1511] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:55.488932 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a0eb7-756c-46ac-bd6b-aea18738c688 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:55.492163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-249a0eb7-756c-46ac-bd6b-aea18738c688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:55.554594 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c1198-b025-48e9-8e1e-ce367cfdc88f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:55.556228 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e84c1198-b025-48e9-8e1e-ce367cfdc88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:55.659678 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:55.707777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a0eb7-756c-46ac-bd6b-aea18738c688 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:55.732426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:55.733085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:55.733363 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:55.734373 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] PATH_INFO: `/v3/projects/090b7837445f44699c852450f2e0e8ee` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:55.735046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=090b7837445f44699c852450f2e0e8ee)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:55.743284 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:55.751332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c1198-b025-48e9-8e1e-ce367cfdc88f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:55.773350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd52b9-f3ab-4116-8e60-fbfbc8e7d938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:55.774063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd52b9-f3ab-4116-8e60-fbfbc8e7d938 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:55.774341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd52b9-f3ab-4116-8e60-fbfbc8e7d938 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:55.774608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd52b9-f3ab-4116-8e60-fbfbc8e7d938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:55.814722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd52b9-f3ab-4116-8e60-fbfbc8e7d938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:55.816896 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd52b9-f3ab-4116-8e60-fbfbc8e7d938 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:55.853914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 756/1512] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:55.890632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '090b7837445f44699c852450f2e0e8ee', 'request_id': 'req-af77070e-fb9b-490e-9ce0-9c2a15820521'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:55.916210 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '090b7837445f44699c852450f2e0e8ee', 'request_id': 'req-af77070e-fb9b-490e-9ce0-9c2a15820521'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:55.943843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '090b7837445f44699c852450f2e0e8ee', 'request_id': 'req-af77070e-fb9b-490e-9ce0-9c2a15820521'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:55.995691 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] The token cache is being invalidate because project 090b7837445f44699c852450f2e0e8ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:55.997579 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:55.999617 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:56.008694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 757/1513] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:55 2022] DELETE /identity/v3/projects/090b7837445f44699c852450f2e0e8ee => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:56.014144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f916041-d7a0-4a35-93b6-e9876cf2a8e5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:56.014724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f916041-d7a0-4a35-93b6-e9876cf2a8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:56.015344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f916041-d7a0-4a35-93b6-e9876cf2a8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:56.076746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2f916041-d7a0-4a35-93b6-e9876cf2a8e5 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:56.079635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2f916041-d7a0-4a35-93b6-e9876cf2a8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:56.310048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:56.311540 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:56.371340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 757/1514] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:56.377711 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd531b5-cee3-4302-9e71-9fc7ec6f0069 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:56.378203 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd531b5-cee3-4302-9e71-9fc7ec6f0069 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:56.378732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd531b5-cee3-4302-9e71-9fc7ec6f0069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:56.426941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4cd531b5-cee3-4302-9e71-9fc7ec6f0069 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:56.430722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4cd531b5-cee3-4302-9e71-9fc7ec6f0069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:56.449240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 758/1515] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:56.454999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd531b5-cee3-4302-9e71-9fc7ec6f0069 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:56.493734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd531b5-cee3-4302-9e71-9fc7ec6f0069 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:56.508663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d883e6-4850-459b-8449-31898ed1665d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:56.509604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d883e6-4850-459b-8449-31898ed1665d admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:56.509902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d883e6-4850-459b-8449-31898ed1665d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:56.510165 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d883e6-4850-459b-8449-31898ed1665d admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:56.511019 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d883e6-4850-459b-8449-31898ed1665d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:56.511157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d883e6-4850-459b-8449-31898ed1665d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:56.512796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d883e6-4850-459b-8449-31898ed1665d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:56.532282 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77070e-fb9b-490e-9ce0-9c2a15820521 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:56.552554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fca9e-950c-4f1e-b94b-cc8447f0fb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:56.553274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fca9e-950c-4f1e-b94b-cc8447f0fb9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:56.553560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fca9e-950c-4f1e-b94b-cc8447f0fb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:56.553880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fca9e-950c-4f1e-b94b-cc8447f0fb9b admin admin] PATH_INFO: `/v3/users/86f1ab1b9e634a91a5884729f1d30a87` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:56.557376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 759/1516] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:56.558115 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fca9e-950c-4f1e-b94b-cc8447f0fb9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=86f1ab1b9e634a91a5884729f1d30a87)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:56.560112 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fca9e-950c-4f1e-b94b-cc8447f0fb9b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:56.561341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d883e6-4850-459b-8449-31898ed1665d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:56.586605 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d883e6-4850-459b-8449-31898ed1665d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:56.595847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa666f-5ece-44cc-bedf-96f6553a76b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:56.597839 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa666f-5ece-44cc-bedf-96f6553a76b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:56.598155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa666f-5ece-44cc-bedf-96f6553a76b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:56.598439 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa666f-5ece-44cc-bedf-96f6553a76b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:56.599914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa666f-5ece-44cc-bedf-96f6553a76b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:56.605482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa666f-5ece-44cc-bedf-96f6553a76b5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:56.679765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4d8fca9e-950c-4f1e-b94b-cc8447f0fb9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86f1ab1b9e634a91a5884729f1d30a87'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:56.685585 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 760/1517] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:56 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:56.691303 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa666f-5ece-44cc-bedf-96f6553a76b5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:56.700237 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4d8fca9e-950c-4f1e-b94b-cc8447f0fb9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86f1ab1b9e634a91a5884729f1d30a87'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:56.700945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4d8fca9e-950c-4f1e-b94b-cc8447f0fb9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86f1ab1b9e634a91a5884729f1d30a87'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:56.751632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4d8fca9e-950c-4f1e-b94b-cc8447f0fb9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86f1ab1b9e634a91a5884729f1d30a87'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:56.769986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4d8fca9e-950c-4f1e-b94b-cc8447f0fb9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86f1ab1b9e634a91a5884729f1d30a87'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:56.777720 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:56.794212 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 758/1518] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:56 2022] DELETE /identity/v3/users/86f1ab1b9e634a91a5884729f1d30a87 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:56.796049 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117a66a-d2eb-4352-8963-703d4bb377d6 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:56.796337 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117a66a-d2eb-4352-8963-703d4bb377d6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:56.796615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117a66a-d2eb-4352-8963-703d4bb377d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:56.841162 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d117a66a-d2eb-4352-8963-703d4bb377d6 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:56.846461 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d117a66a-d2eb-4352-8963-703d4bb377d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:56.869891 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:56.896385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa666f-5ece-44cc-bedf-96f6553a76b5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:56.973599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 759/1519] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:57.002266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117a66a-d2eb-4352-8963-703d4bb377d6 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:57.004742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d117a66a-d2eb-4352-8963-703d4bb377d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:57.032209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ce3cd-108d-4445-ba4d-f9909ffb6ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:57.033223 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ce3cd-108d-4445-ba4d-f9909ffb6ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.033682 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ce3cd-108d-4445-ba4d-f9909ffb6ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.034374 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ce3cd-108d-4445-ba4d-f9909ffb6ea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.035656 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ce3cd-108d-4445-ba4d-f9909ffb6ea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:57.037844 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ce3cd-108d-4445-ba4d-f9909ffb6ea9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:57.070102 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 761/1520] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:56 2022] POST /identity/v3/users => generated 294 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:57.073861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ce3cd-108d-4445-ba4d-f9909ffb6ea9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:57.077702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-502ce3cd-108d-4445-ba4d-f9909ffb6ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:57.099347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ce3cd-108d-4445-ba4d-f9909ffb6ea9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:57.105202 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117a66a-d2eb-4352-8963-703d4bb377d6 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:57.105491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade49736-fb21-42eb-b0d2-b9ac46c29061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:57.106227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade49736-fb21-42eb-b0d2-b9ac46c29061 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.106623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade49736-fb21-42eb-b0d2-b9ac46c29061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.107014 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade49736-fb21-42eb-b0d2-b9ac46c29061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.107844 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade49736-fb21-42eb-b0d2-b9ac46c29061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:57.109578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade49736-fb21-42eb-b0d2-b9ac46c29061 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:57.114267 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e0682-a96a-4ec5-aa58-b22a81696102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:57.126236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 762/1521] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:24:57.127385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade49736-fb21-42eb-b0d2-b9ac46c29061 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:57.128060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e0682-a96a-4ec5-aa58-b22a81696102 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.128478 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e0682-a96a-4ec5-aa58-b22a81696102 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.128753 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e0682-a96a-4ec5-aa58-b22a81696102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.138561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade49736-fb21-42eb-b0d2-b9ac46c29061 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:57.150707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e0682-a96a-4ec5-aa58-b22a81696102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:57.152871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e0682-a96a-4ec5-aa58-b22a81696102 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:57.173810 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432403ab-c9a4-4659-a8b7-88e77c27d821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:57.174457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432403ab-c9a4-4659-a8b7-88e77c27d821 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.174735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432403ab-c9a4-4659-a8b7-88e77c27d821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.175002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432403ab-c9a4-4659-a8b7-88e77c27d821 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.176285 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432403ab-c9a4-4659-a8b7-88e77c27d821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:57.178549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432403ab-c9a4-4659-a8b7-88e77c27d821 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:57.178956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 760/1522] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:24:57.195614 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 763/1523] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:57.197116 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e0682-a96a-4ec5-aa58-b22a81696102 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:57.210349 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e0682-a96a-4ec5-aa58-b22a81696102 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:57.214736 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fa69c-0811-40af-9726-70da7c9f495c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:57.215502 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fa69c-0811-40af-9726-70da7c9f495c admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.215786 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fa69c-0811-40af-9726-70da7c9f495c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.216444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fa69c-0811-40af-9726-70da7c9f495c admin admin] PATH_INFO: `/v3/projects/d718a5e62d844d25865c2c6341c428ea/users/9a7a4ec966dd41ad84ebc0c114941062/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.232460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fa69c-0811-40af-9726-70da7c9f495c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d718a5e62d844d25865c2c6341c428ea, user_id=9a7a4ec966dd41ad84ebc0c114941062, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:57.235287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fa69c-0811-40af-9726-70da7c9f495c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:57.252079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 761/1524] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:57 2022] PUT /identity/v3/projects/d718a5e62d844d25865c2c6341c428ea/users/9a7a4ec966dd41ad84ebc0c114941062/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:24:57.265084 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d0d4b-29e4-4b3a-974f-9407bbbac5b9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.265484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d0d4b-29e4-4b3a-974f-9407bbbac5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.265767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d0d4b-29e4-4b3a-974f-9407bbbac5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.305419 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-441d0d4b-29e4-4b3a-974f-9407bbbac5b9 None None] MFA Rules not processed for user `9a7a4ec966dd41ad84ebc0c114941062`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:57.307313 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-441d0d4b-29e4-4b3a-974f-9407bbbac5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:57.387322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 764/1525] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:57.391231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fa69c-0811-40af-9726-70da7c9f495c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:57.466379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fa69c-0811-40af-9726-70da7c9f495c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:57.471016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ab408-d0a2-49f2-b7ea-83ddd887349c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:57.471669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ab408-d0a2-49f2-b7ea-83ddd887349c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.471949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ab408-d0a2-49f2-b7ea-83ddd887349c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.472747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ab408-d0a2-49f2-b7ea-83ddd887349c admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.473393 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ab408-d0a2-49f2-b7ea-83ddd887349c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:57.478075 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ab408-d0a2-49f2-b7ea-83ddd887349c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:57.524261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 762/1526] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:57 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:57.525693 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d0d4b-29e4-4b3a-974f-9407bbbac5b9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:57.539864 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d0d4b-29e4-4b3a-974f-9407bbbac5b9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:57.612408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:57.624520 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cca53c-64d4-4e61-bc67-abaee357917b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:57.625524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cca53c-64d4-4e61-bc67-abaee357917b admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.625942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cca53c-64d4-4e61-bc67-abaee357917b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.626450 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cca53c-64d4-4e61-bc67-abaee357917b admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.627128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cca53c-64d4-4e61-bc67-abaee357917b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:57.631127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cca53c-64d4-4e61-bc67-abaee357917b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:57.639884 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ab408-d0a2-49f2-b7ea-83ddd887349c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:57.642399 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-194ab408-d0a2-49f2-b7ea-83ddd887349c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:57.662219 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:57.667957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 765/1527] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:57 2022] POST /identity/v3/users => generated 299 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:57.669051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cca53c-64d4-4e61-bc67-abaee357917b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:57.675954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ab408-d0a2-49f2-b7ea-83ddd887349c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:57.680923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cca53c-64d4-4e61-bc67-abaee357917b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:57.682884 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:57.685714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f292b93-349e-4a29-922d-445814f36628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:57.686408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f292b93-349e-4a29-922d-445814f36628 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.686752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f292b93-349e-4a29-922d-445814f36628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.687171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f292b93-349e-4a29-922d-445814f36628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.687958 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f292b93-349e-4a29-922d-445814f36628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:57.688067 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.688457 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.688744 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] PATH_INFO: `/v3/projects/e84585f806fe405b9981a3c1f6e91263` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.689806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f292b93-349e-4a29-922d-445814f36628 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:57.690082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e84585f806fe405b9981a3c1f6e91263)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:57.692130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:57.704882 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 766/1528] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:57.708815 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f292b93-349e-4a29-922d-445814f36628 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:57.728966 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f292b93-349e-4a29-922d-445814f36628 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:57.729216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829b528-3195-4720-b9a9-ef72cbd6ae0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:57.730142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829b528-3195-4720-b9a9-ef72cbd6ae0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.730511 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829b528-3195-4720-b9a9-ef72cbd6ae0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.730923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829b528-3195-4720-b9a9-ef72cbd6ae0d admin admin] PATH_INFO: `/v3/projects/855aa07345b4494b8f5381563a88db49/users/f1b0ad1a4c4e42c7b8f4aa620057bfeb/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.756485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829b528-3195-4720-b9a9-ef72cbd6ae0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=855aa07345b4494b8f5381563a88db49, user_id=f1b0ad1a4c4e42c7b8f4aa620057bfeb, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:57.761251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829b528-3195-4720-b9a9-ef72cbd6ae0d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:57.764770 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e84585f806fe405b9981a3c1f6e91263', 'request_id': 'req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:57.772341 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e84585f806fe405b9981a3c1f6e91263', 'request_id': 'req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:57.780991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 767/1529] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:57 2022] PUT /identity/v3/projects/855aa07345b4494b8f5381563a88db49/users/f1b0ad1a4c4e42c7b8f4aa620057bfeb/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:24:57.782774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e84585f806fe405b9981a3c1f6e91263', 'request_id': 'req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:57.785869 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829b528-3195-4720-b9a9-ef72cbd6ae0d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:57.801522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] The token cache is being invalidate because project e84585f806fe405b9981a3c1f6e91263 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:57.801967 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:57.803365 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:57.806886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 763/1530] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:57 2022] DELETE /identity/v3/projects/e84585f806fe405b9981a3c1f6e91263 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:24:57.810825 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:57.812712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:57.866645 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829b528-3195-4720-b9a9-ef72cbd6ae0d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:57.876859 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b88825-a98a-42a1-80e5-f9900eda7683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:57.877678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b88825-a98a-42a1-80e5-f9900eda7683 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.878056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b88825-a98a-42a1-80e5-f9900eda7683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.878461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b88825-a98a-42a1-80e5-f9900eda7683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.879496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b88825-a98a-42a1-80e5-f9900eda7683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:57.881662 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b88825-a98a-42a1-80e5-f9900eda7683 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:57.894508 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 768/1531] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:57.898758 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b88825-a98a-42a1-80e5-f9900eda7683 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:57.902435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e5b88825-a98a-42a1-80e5-f9900eda7683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:57.905562 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdde840-16bf-4e1e-9b27-cdd9a3000e2a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:57.910329 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf02b9-dfa8-4879-82f7-b996c9ffe4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:57.911059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf02b9-dfa8-4879-82f7-b996c9ffe4af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.911439 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf02b9-dfa8-4879-82f7-b996c9ffe4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.911781 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf02b9-dfa8-4879-82f7-b996c9ffe4af admin admin] PATH_INFO: `/v3/users/b2d10151fb1849a8bc602b7a70bcc891` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.912999 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf02b9-dfa8-4879-82f7-b996c9ffe4af admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2d10151fb1849a8bc602b7a70bcc891)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:57.914998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf02b9-dfa8-4879-82f7-b996c9ffe4af admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:57.922804 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b88825-a98a-42a1-80e5-f9900eda7683 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:57.935156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5235e-9443-47ba-ba7b-9e6b080f6216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:57.935856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5235e-9443-47ba-ba7b-9e6b080f6216 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.936137 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5235e-9443-47ba-ba7b-9e6b080f6216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.936466 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5235e-9443-47ba-ba7b-9e6b080f6216 admin admin] PATH_INFO: `/v3/projects/855aa07345b4494b8f5381563a88db49/users/f1b0ad1a4c4e42c7b8f4aa620057bfeb/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.941785 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5235e-9443-47ba-ba7b-9e6b080f6216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=855aa07345b4494b8f5381563a88db49, user_id=f1b0ad1a4c4e42c7b8f4aa620057bfeb, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:57.949778 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5235e-9443-47ba-ba7b-9e6b080f6216 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:57.973871 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 769/1532] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:57 2022] PUT /identity/v3/projects/855aa07345b4494b8f5381563a88db49/users/f1b0ad1a4c4e42c7b8f4aa620057bfeb/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:24:57.975742 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c5fcc-9a82-4127-9ba1-cee100cd5228 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:57.976041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c5fcc-9a82-4127-9ba1-cee100cd5228 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:57.976301 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c5fcc-9a82-4127-9ba1-cee100cd5228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:57.983455 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-50bf02b9-dfa8-4879-82f7-b996c9ffe4af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2d10151fb1849a8bc602b7a70bcc891'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:58.006299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-50bf02b9-dfa8-4879-82f7-b996c9ffe4af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2d10151fb1849a8bc602b7a70bcc891'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:58.007370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-50bf02b9-dfa8-4879-82f7-b996c9ffe4af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2d10151fb1849a8bc602b7a70bcc891'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:58.018369 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-698c5fcc-9a82-4127-9ba1-cee100cd5228 None None] MFA Rules not processed for user `f1b0ad1a4c4e42c7b8f4aa620057bfeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:58.020771 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-698c5fcc-9a82-4127-9ba1-cee100cd5228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:58.026560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-50bf02b9-dfa8-4879-82f7-b996c9ffe4af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2d10151fb1849a8bc602b7a70bcc891'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:58.036042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-50bf02b9-dfa8-4879-82f7-b996c9ffe4af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2d10151fb1849a8bc602b7a70bcc891'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:58.048560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 764/1533] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:57 2022] DELETE /identity/v3/users/b2d10151fb1849a8bc602b7a70bcc891 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:24:58.110132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 770/1534] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:24:58.122656 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf02b9-dfa8-4879-82f7-b996c9ffe4af admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:58.124304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-50bf02b9-dfa8-4879-82f7-b996c9ffe4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:58.136573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:58.199562 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf02b9-dfa8-4879-82f7-b996c9ffe4af admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:58.204128 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96983e7c-60d5-42f2-b7ab-28173735c585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:58.204788 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96983e7c-60d5-42f2-b7ab-28173735c585 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:58.205159 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96983e7c-60d5-42f2-b7ab-28173735c585 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:58.205446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96983e7c-60d5-42f2-b7ab-28173735c585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:58.277461 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96983e7c-60d5-42f2-b7ab-28173735c585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:58.279093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96983e7c-60d5-42f2-b7ab-28173735c585 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:58.289511 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 765/1535] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:58.497633 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c5fcc-9a82-4127-9ba1-cee100cd5228 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:58.499279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-698c5fcc-9a82-4127-9ba1-cee100cd5228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:58.531793 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:58.581236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c5fcc-9a82-4127-9ba1-cee100cd5228 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:58.648190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e662d2-f6f5-4e0a-8856-aeaaf12de291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:58.648761 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e662d2-f6f5-4e0a-8856-aeaaf12de291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:58.649044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e662d2-f6f5-4e0a-8856-aeaaf12de291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:58.649346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e662d2-f6f5-4e0a-8856-aeaaf12de291 admin admin] PATH_INFO: `/v3/projects/8d24ef2a62564a749134cc89e67a6d37` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:58.650637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e662d2-f6f5-4e0a-8856-aeaaf12de291 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d24ef2a62564a749134cc89e67a6d37)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:58.652755 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e662d2-f6f5-4e0a-8856-aeaaf12de291 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:58.709295 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-15e662d2-f6f5-4e0a-8856-aeaaf12de291 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d24ef2a62564a749134cc89e67a6d37', 'request_id': 'req-15e662d2-f6f5-4e0a-8856-aeaaf12de291'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:58.714812 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-15e662d2-f6f5-4e0a-8856-aeaaf12de291 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d24ef2a62564a749134cc89e67a6d37', 'request_id': 'req-15e662d2-f6f5-4e0a-8856-aeaaf12de291'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:58.721033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-15e662d2-f6f5-4e0a-8856-aeaaf12de291 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d24ef2a62564a749134cc89e67a6d37', 'request_id': 'req-15e662d2-f6f5-4e0a-8856-aeaaf12de291'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:58.736181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-15e662d2-f6f5-4e0a-8856-aeaaf12de291 admin admin] The token cache is being invalidate because project 8d24ef2a62564a749134cc89e67a6d37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:24:58.736465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-15e662d2-f6f5-4e0a-8856-aeaaf12de291 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:58.737131 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-15e662d2-f6f5-4e0a-8856-aeaaf12de291 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:58.739355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 771/1536] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:24:58 2022] DELETE /identity/v3/projects/8d24ef2a62564a749134cc89e67a6d37 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:24:58.744840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445468f7-fd6f-4d79-8a40-f3023a38f47d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:58.745276 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445468f7-fd6f-4d79-8a40-f3023a38f47d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:58.745534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445468f7-fd6f-4d79-8a40-f3023a38f47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:58.765727 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-445468f7-fd6f-4d79-8a40-f3023a38f47d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:58.767013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-445468f7-fd6f-4d79-8a40-f3023a38f47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:58.833200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 766/1537] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:58.842756 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f02c0-8679-4625-a8e9-4346d3007f43 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:58.843036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f02c0-8679-4625-a8e9-4346d3007f43 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:58.843369 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f02c0-8679-4625-a8e9-4346d3007f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:58.863838 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ac7f02c0-8679-4625-a8e9-4346d3007f43 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:58.865048 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ac7f02c0-8679-4625-a8e9-4346d3007f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:58.875200 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 772/1538] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:58.878167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445468f7-fd6f-4d79-8a40-f3023a38f47d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:58.889003 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445468f7-fd6f-4d79-8a40-f3023a38f47d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:58.894459 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de49f0-b58c-4759-9dbf-1a4be39c16c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:58.895092 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de49f0-b58c-4759-9dbf-1a4be39c16c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:58.895379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de49f0-b58c-4759-9dbf-1a4be39c16c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:58.895646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de49f0-b58c-4759-9dbf-1a4be39c16c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:58.896373 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de49f0-b58c-4759-9dbf-1a4be39c16c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:24:58.896679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de49f0-b58c-4759-9dbf-1a4be39c16c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:58.898304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de49f0-b58c-4759-9dbf-1a4be39c16c0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:58.911397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 767/1539] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:24:58.915631 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f02c0-8679-4625-a8e9-4346d3007f43 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:58.924799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f02c0-8679-4625-a8e9-4346d3007f43 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:58.933283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259472f-5364-4b51-97a2-34f854deece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:58.933900 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259472f-5364-4b51-97a2-34f854deece4 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:58.934171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259472f-5364-4b51-97a2-34f854deece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:58.934438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259472f-5364-4b51-97a2-34f854deece4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:58.935116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a259472f-5364-4b51-97a2-34f854deece4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:58.936987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a259472f-5364-4b51-97a2-34f854deece4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:58.946386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de49f0-b58c-4759-9dbf-1a4be39c16c0 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:58.948198 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-34de49f0-b58c-4759-9dbf-1a4be39c16c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:58.961917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de49f0-b58c-4759-9dbf-1a4be39c16c0 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:58.967129 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 773/1540] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:58 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:58.967261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541af711-0509-478b-9c69-5889915a7b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:58.967404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541af711-0509-478b-9c69-5889915a7b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:58.967687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541af711-0509-478b-9c69-5889915a7b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:58.967958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541af711-0509-478b-9c69-5889915a7b18 admin admin] PATH_INFO: `/v3/users/9a7a4ec966dd41ad84ebc0c114941062` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:58.969155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541af711-0509-478b-9c69-5889915a7b18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a7a4ec966dd41ad84ebc0c114941062)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:58.969282 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259472f-5364-4b51-97a2-34f854deece4 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:58.971168 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541af711-0509-478b-9c69-5889915a7b18 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:59.008410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:59.024007 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:59.031880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-541af711-0509-478b-9c69-5889915a7b18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a7a4ec966dd41ad84ebc0c114941062'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:59.042226 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-541af711-0509-478b-9c69-5889915a7b18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a7a4ec966dd41ad84ebc0c114941062'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:59.043574 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-541af711-0509-478b-9c69-5889915a7b18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a7a4ec966dd41ad84ebc0c114941062'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:59.060721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259472f-5364-4b51-97a2-34f854deece4 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:59.067585 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382812f-5756-40bb-a3e1-618f9fce2026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:59.068485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382812f-5756-40bb-a3e1-618f9fce2026 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:59.068985 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382812f-5756-40bb-a3e1-618f9fce2026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:59.069420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382812f-5756-40bb-a3e1-618f9fce2026 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:59.070220 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382812f-5756-40bb-a3e1-618f9fce2026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:59.072592 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382812f-5756-40bb-a3e1-618f9fce2026 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:59.078759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-541af711-0509-478b-9c69-5889915a7b18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a7a4ec966dd41ad84ebc0c114941062'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:59.097105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-541af711-0509-478b-9c69-5889915a7b18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a7a4ec966dd41ad84ebc0c114941062'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:24:59.108008 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 774/1541] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:58 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:24:59.112230 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 768/1542] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:24:58 2022] DELETE /identity/v3/users/9a7a4ec966dd41ad84ebc0c114941062 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:24:59.112579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382812f-5756-40bb-a3e1-618f9fce2026 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:59.116796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2382812f-5756-40bb-a3e1-618f9fce2026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:59.120138 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdd00c-face-493b-b974-996dda56ee49 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:59.120590 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdd00c-face-493b-b974-996dda56ee49 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:59.121060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdd00c-face-493b-b974-996dda56ee49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:59.142790 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382812f-5756-40bb-a3e1-618f9fce2026 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:59.151057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-50fdd00c-face-493b-b974-996dda56ee49 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:59.152681 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-50fdd00c-face-493b-b974-996dda56ee49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:59.279155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 769/1543] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:59.290986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdd00c-face-493b-b974-996dda56ee49 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:59.292030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27308296-22b5-4933-873c-e4210444e3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:59.292658 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27308296-22b5-4933-873c-e4210444e3de admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:59.292934 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27308296-22b5-4933-873c-e4210444e3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:59.293207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27308296-22b5-4933-873c-e4210444e3de admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:59.293347 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-50fdd00c-face-493b-b974-996dda56ee49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:59.293864 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27308296-22b5-4933-873c-e4210444e3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:59.295772 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27308296-22b5-4933-873c-e4210444e3de admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:59.307500 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 775/1544] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:59.311860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27308296-22b5-4933-873c-e4210444e3de admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:59.325683 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27308296-22b5-4933-873c-e4210444e3de admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:59.337882 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5b15d-cd4e-4a7e-8d4c-567d72cdbf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:59.339276 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5b15d-cd4e-4a7e-8d4c-567d72cdbf1f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:59.341117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5b15d-cd4e-4a7e-8d4c-567d72cdbf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:59.341465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5b15d-cd4e-4a7e-8d4c-567d72cdbf1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:59.342186 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5b15d-cd4e-4a7e-8d4c-567d72cdbf1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:59.344543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5b15d-cd4e-4a7e-8d4c-567d72cdbf1f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:59.376703 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 776/1545] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:24:59.381372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5b15d-cd4e-4a7e-8d4c-567d72cdbf1f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:59.417726 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5b15d-cd4e-4a7e-8d4c-567d72cdbf1f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:59.423362 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdd00c-face-493b-b974-996dda56ee49 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:59.424699 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a02e48-775c-45ad-a2a8-a377b43b6126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:59.426184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a02e48-775c-45ad-a2a8-a377b43b6126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:59.426552 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a02e48-775c-45ad-a2a8-a377b43b6126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:59.426940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a02e48-775c-45ad-a2a8-a377b43b6126 admin admin] PATH_INFO: `/v3/projects/3d4f62d58dd7415e88e4ff1430d8ae9b/users/d2aff3bf91dc4da8b44b7866f1ff62f2/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:59.427980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39a1ef-12b0-4517-a213-11dfd0ba0292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:59.428828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39a1ef-12b0-4517-a213-11dfd0ba0292 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:59.429601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39a1ef-12b0-4517-a213-11dfd0ba0292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:59.429601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39a1ef-12b0-4517-a213-11dfd0ba0292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:59.444925 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a02e48-775c-45ad-a2a8-a377b43b6126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d4f62d58dd7415e88e4ff1430d8ae9b, user_id=d2aff3bf91dc4da8b44b7866f1ff62f2, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:59.448231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a02e48-775c-45ad-a2a8-a377b43b6126 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:59.452683 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39a1ef-12b0-4517-a213-11dfd0ba0292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:59.454630 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39a1ef-12b0-4517-a213-11dfd0ba0292 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:59.476162 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 770/1546] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:24:59.524499 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 777/1547] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:24:59 2022] PUT /identity/v3/projects/3d4f62d58dd7415e88e4ff1430d8ae9b/users/d2aff3bf91dc4da8b44b7866f1ff62f2/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:24:59.531352 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730b8af-abc9-468e-abb8-9b63ec23188b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:59.531647 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730b8af-abc9-468e-abb8-9b63ec23188b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:59.531917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730b8af-abc9-468e-abb8-9b63ec23188b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:59.609726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c730b8af-abc9-468e-abb8-9b63ec23188b None None] MFA Rules not processed for user `d2aff3bf91dc4da8b44b7866f1ff62f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:24:59.611229 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c730b8af-abc9-468e-abb8-9b63ec23188b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:24:59.780550 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:24:59.853746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 771/1548] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:24:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:24:59.858515 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a02e48-775c-45ad-a2a8-a377b43b6126 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:24:59.945640 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a02e48-775c-45ad-a2a8-a377b43b6126 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:24:59.951317 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282305de-b674-4bec-b261-2bfd3f431ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:24:59.952006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282305de-b674-4bec-b261-2bfd3f431ace admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:24:59.952395 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282305de-b674-4bec-b261-2bfd3f431ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:24:59.952683 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282305de-b674-4bec-b261-2bfd3f431ace admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:24:59.953381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282305de-b674-4bec-b261-2bfd3f431ace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:24:59.955610 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282305de-b674-4bec-b261-2bfd3f431ace admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:24:59.996167 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 778/1549] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:24:59 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:24:59.997708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730b8af-abc9-468e-abb8-9b63ec23188b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:00.012995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730b8af-abc9-468e-abb8-9b63ec23188b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:00.101212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282305de-b674-4bec-b261-2bfd3f431ace admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:00.104279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-282305de-b674-4bec-b261-2bfd3f431ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:00.119802 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:00.147129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d5c97-f016-416f-9990-910cf748c0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:00.147788 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d5c97-f016-416f-9990-910cf748c0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:00.148064 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d5c97-f016-416f-9990-910cf748c0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:00.148385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d5c97-f016-416f-9990-910cf748c0d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:00.149136 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d5c97-f016-416f-9990-910cf748c0d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:00.153935 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d5c97-f016-416f-9990-910cf748c0d9 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:00.205425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 772/1550] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:24:59 2022] POST /identity/v3/users => generated 300 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:00.207001 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d5c97-f016-416f-9990-910cf748c0d9 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:00.228395 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d5c97-f016-416f-9990-910cf748c0d9 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:00.235328 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa055485-1430-4d7e-ab2d-da1ccf8e0486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:00.236746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa055485-1430-4d7e-ab2d-da1ccf8e0486 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:00.237302 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa055485-1430-4d7e-ab2d-da1ccf8e0486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:00.238412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa055485-1430-4d7e-ab2d-da1ccf8e0486 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:00.239919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa055485-1430-4d7e-ab2d-da1ccf8e0486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:00.243659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa055485-1430-4d7e-ab2d-da1ccf8e0486 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:00.263676 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 773/1551] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:00.268000 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa055485-1430-4d7e-ab2d-da1ccf8e0486 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:00.294302 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282305de-b674-4bec-b261-2bfd3f431ace admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:00.301625 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa055485-1430-4d7e-ab2d-da1ccf8e0486 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:00.304946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:00.305773 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:00.306057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:00.306324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] PATH_INFO: `/v3/projects/d718a5e62d844d25865c2c6341c428ea` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:00.307904 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d718a5e62d844d25865c2c6341c428ea)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:00.311431 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598a6d2-d315-4320-a965-942fa4f47f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:00.314576 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598a6d2-d315-4320-a965-942fa4f47f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:00.314996 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:00.315386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598a6d2-d315-4320-a965-942fa4f47f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:00.316602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598a6d2-d315-4320-a965-942fa4f47f1b admin admin] PATH_INFO: `/v3/projects/1be1e0ac2bbe4d83a7d22b291d6bd201/users/9e4489c178404a069215f6bc2b4b7d37/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:00.360430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598a6d2-d315-4320-a965-942fa4f47f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1be1e0ac2bbe4d83a7d22b291d6bd201, user_id=9e4489c178404a069215f6bc2b4b7d37, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:00.365635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598a6d2-d315-4320-a965-942fa4f47f1b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:00.398482 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 774/1552] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:00 2022] PUT /identity/v3/projects/1be1e0ac2bbe4d83a7d22b291d6bd201/users/9e4489c178404a069215f6bc2b4b7d37/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:25:00.402898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598a6d2-d315-4320-a965-942fa4f47f1b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:00.468826 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd718a5e62d844d25865c2c6341c428ea', 'request_id': 'req-32ab399e-c349-4b90-826e-e1376d552837'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:00.481905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd718a5e62d844d25865c2c6341c428ea', 'request_id': 'req-32ab399e-c349-4b90-826e-e1376d552837'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:00.503485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd718a5e62d844d25865c2c6341c428ea', 'request_id': 'req-32ab399e-c349-4b90-826e-e1376d552837'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:00.524255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] The token cache is being invalidate because project d718a5e62d844d25865c2c6341c428ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:00.524546 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:00.525878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:00.532361 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 779/1553] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:00 2022] DELETE /identity/v3/projects/d718a5e62d844d25865c2c6341c428ea => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:00.533566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:00.535899 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:00.577893 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598a6d2-d315-4320-a965-942fa4f47f1b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:00.636475 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab399e-c349-4b90-826e-e1376d552837 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:00.643483 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df7d5f-0561-401b-a903-cc367877b8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:00.645005 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df7d5f-0561-401b-a903-cc367877b8f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:00.646265 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df7d5f-0561-401b-a903-cc367877b8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:00.646933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df7d5f-0561-401b-a903-cc367877b8f2 admin admin] PATH_INFO: `/v3/users/f1b0ad1a4c4e42c7b8f4aa620057bfeb` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:00.648808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df7d5f-0561-401b-a903-cc367877b8f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b0ad1a4c4e42c7b8f4aa620057bfeb)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:00.654365 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df7d5f-0561-401b-a903-cc367877b8f2 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:00.670890 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:00.707434 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddb5b4-6b8b-4f88-8388-5f2cd1ce2ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:00.708702 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddb5b4-6b8b-4f88-8388-5f2cd1ce2ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:00.709029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddb5b4-6b8b-4f88-8388-5f2cd1ce2ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:00.709412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddb5b4-6b8b-4f88-8388-5f2cd1ce2ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:00.710272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ddb5b4-6b8b-4f88-8388-5f2cd1ce2ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:00.712826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ddb5b4-6b8b-4f88-8388-5f2cd1ce2ad0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:00.746844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 775/1554] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:00.747566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:00.747984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddb5b4-6b8b-4f88-8388-5f2cd1ce2ad0 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:00.750064 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f6ddb5b4-6b8b-4f88-8388-5f2cd1ce2ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:00.760974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b5df7d5f-0561-401b-a903-cc367877b8f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b0ad1a4c4e42c7b8f4aa620057bfeb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:00.766200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddb5b4-6b8b-4f88-8388-5f2cd1ce2ad0 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:00.773757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53017d9-0ba3-4f1d-8ee0-bfb7234fceb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:00.774565 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53017d9-0ba3-4f1d-8ee0-bfb7234fceb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:00.775017 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53017d9-0ba3-4f1d-8ee0-bfb7234fceb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:00.777648 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53017d9-0ba3-4f1d-8ee0-bfb7234fceb2 admin admin] PATH_INFO: `/v3/projects/1be1e0ac2bbe4d83a7d22b291d6bd201/users/9e4489c178404a069215f6bc2b4b7d37/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:00.780614 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53017d9-0ba3-4f1d-8ee0-bfb7234fceb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1be1e0ac2bbe4d83a7d22b291d6bd201, user_id=9e4489c178404a069215f6bc2b4b7d37, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:00.783969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53017d9-0ba3-4f1d-8ee0-bfb7234fceb2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:00.786525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b5df7d5f-0561-401b-a903-cc367877b8f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b0ad1a4c4e42c7b8f4aa620057bfeb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:00.804460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 776/1555] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:00 2022] PUT /identity/v3/projects/1be1e0ac2bbe4d83a7d22b291d6bd201/users/9e4489c178404a069215f6bc2b4b7d37/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:00.813138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b5df7d5f-0561-401b-a903-cc367877b8f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b0ad1a4c4e42c7b8f4aa620057bfeb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:00.820706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6171925-07ad-48a8-9e77-0359a6a589f8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:00.820706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6171925-07ad-48a8-9e77-0359a6a589f8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:00.820706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6171925-07ad-48a8-9e77-0359a6a589f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:00.845040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b5df7d5f-0561-401b-a903-cc367877b8f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b0ad1a4c4e42c7b8f4aa620057bfeb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:00.845875 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b5df7d5f-0561-401b-a903-cc367877b8f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b0ad1a4c4e42c7b8f4aa620057bfeb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:00.863401 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 780/1556] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:00 2022] DELETE /identity/v3/users/f1b0ad1a4c4e42c7b8f4aa620057bfeb => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:00.881262 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d6171925-07ad-48a8-9e77-0359a6a589f8 None None] MFA Rules not processed for user `9e4489c178404a069215f6bc2b4b7d37`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:00.882498 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d6171925-07ad-48a8-9e77-0359a6a589f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:00.965643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 777/1557] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:00.977684 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df7d5f-0561-401b-a903-cc367877b8f2 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:00.979192 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b5df7d5f-0561-401b-a903-cc367877b8f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:01.040089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df7d5f-0561-401b-a903-cc367877b8f2 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:01.043787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639fdfa-f20a-4abe-be57-ab43eeda92a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:01.044772 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639fdfa-f20a-4abe-be57-ab43eeda92a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:01.045047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639fdfa-f20a-4abe-be57-ab43eeda92a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:01.045343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639fdfa-f20a-4abe-be57-ab43eeda92a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:01.055779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7639fdfa-f20a-4abe-be57-ab43eeda92a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:01.057355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7639fdfa-f20a-4abe-be57-ab43eeda92a9 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:01.079761 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 781/1558] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:01.230335 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6171925-07ad-48a8-9e77-0359a6a589f8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:01.231800 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d6171925-07ad-48a8-9e77-0359a6a589f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:01.266969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639fdfa-f20a-4abe-be57-ab43eeda92a9 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:01.268677 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7639fdfa-f20a-4abe-be57-ab43eeda92a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:01.303741 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6171925-07ad-48a8-9e77-0359a6a589f8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:01.308220 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbf1d0-b6d4-4ef0-80b5-29c55001a6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:01.308856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbf1d0-b6d4-4ef0-80b5-29c55001a6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:01.309128 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbf1d0-b6d4-4ef0-80b5-29c55001a6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:01.309397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbf1d0-b6d4-4ef0-80b5-29c55001a6cd admin admin] PATH_INFO: `/v3/users/d2aff3bf91dc4da8b44b7866f1ff62f2` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:01.310499 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbf1d0-b6d4-4ef0-80b5-29c55001a6cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2aff3bf91dc4da8b44b7866f1ff62f2)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:01.312441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbf1d0-b6d4-4ef0-80b5-29c55001a6cd admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:01.350655 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639fdfa-f20a-4abe-be57-ab43eeda92a9 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:01.355765 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63021a36-fc23-47e0-ace1-66284d7db41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:01.356412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63021a36-fc23-47e0-ace1-66284d7db41b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:01.357216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63021a36-fc23-47e0-ace1-66284d7db41b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:01.357669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63021a36-fc23-47e0-ace1-66284d7db41b admin admin] PATH_INFO: `/v3/projects/855aa07345b4494b8f5381563a88db49` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:01.359288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63021a36-fc23-47e0-ace1-66284d7db41b admin admin] RBAC: Authorizing `identity:delete_project(project_id=855aa07345b4494b8f5381563a88db49)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:01.359668 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-60cbf1d0-b6d4-4ef0-80b5-29c55001a6cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2aff3bf91dc4da8b44b7866f1ff62f2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:01.361254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63021a36-fc23-47e0-ace1-66284d7db41b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:01.369528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-60cbf1d0-b6d4-4ef0-80b5-29c55001a6cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2aff3bf91dc4da8b44b7866f1ff62f2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:01.370312 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-60cbf1d0-b6d4-4ef0-80b5-29c55001a6cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2aff3bf91dc4da8b44b7866f1ff62f2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:01.375843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:01.385532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-60cbf1d0-b6d4-4ef0-80b5-29c55001a6cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2aff3bf91dc4da8b44b7866f1ff62f2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:01.393607 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-60cbf1d0-b6d4-4ef0-80b5-29c55001a6cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2aff3bf91dc4da8b44b7866f1ff62f2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:01.400839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 778/1559] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:01 2022] DELETE /identity/v3/users/d2aff3bf91dc4da8b44b7866f1ff62f2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:01.405503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b16b0-930e-4e8a-a83b-e6181d1e8a0e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:01.405816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b16b0-930e-4e8a-a83b-e6181d1e8a0e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:01.406105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b16b0-930e-4e8a-a83b-e6181d1e8a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:01.412561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-63021a36-fc23-47e0-ace1-66284d7db41b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '855aa07345b4494b8f5381563a88db49', 'request_id': 'req-63021a36-fc23-47e0-ace1-66284d7db41b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:01.418329 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-63021a36-fc23-47e0-ace1-66284d7db41b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '855aa07345b4494b8f5381563a88db49', 'request_id': 'req-63021a36-fc23-47e0-ace1-66284d7db41b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:01.419464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:01.426883 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7e5b16b0-930e-4e8a-a83b-e6181d1e8a0e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:01.427974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-63021a36-fc23-47e0-ace1-66284d7db41b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '855aa07345b4494b8f5381563a88db49', 'request_id': 'req-63021a36-fc23-47e0-ace1-66284d7db41b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:01.428112 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7e5b16b0-930e-4e8a-a83b-e6181d1e8a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:01.442236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-63021a36-fc23-47e0-ace1-66284d7db41b admin admin] The token cache is being invalidate because project 855aa07345b4494b8f5381563a88db49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:01.442525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-63021a36-fc23-47e0-ace1-66284d7db41b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:01.443200 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-63021a36-fc23-47e0-ace1-66284d7db41b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:01.444721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 782/1560] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:01 2022] DELETE /identity/v3/projects/855aa07345b4494b8f5381563a88db49 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:01.449448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14da53a-940b-4827-a42f-f69ae99dc4a3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:01.449758 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14da53a-940b-4827-a42f-f69ae99dc4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:01.450033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14da53a-940b-4827-a42f-f69ae99dc4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:01.469873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a14da53a-940b-4827-a42f-f69ae99dc4a3 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:01.471158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a14da53a-940b-4827-a42f-f69ae99dc4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:01.492468 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 779/1561] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:25:01.503722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b16b0-930e-4e8a-a83b-e6181d1e8a0e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:01.505103 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7e5b16b0-930e-4e8a-a83b-e6181d1e8a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:01.535550 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 783/1562] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:25:01.543369 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483d672-ae38-4486-a881-c763c7b30ff3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:01.543656 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483d672-ae38-4486-a881-c763c7b30ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:01.543930 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483d672-ae38-4486-a881-c763c7b30ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:01.562471 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b16b0-930e-4e8a-a83b-e6181d1e8a0e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:01.564034 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1483d672-ae38-4486-a881-c763c7b30ff3 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:01.565232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1483d672-ae38-4486-a881-c763c7b30ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:01.566218 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674ba96-8f62-490d-bff0-872a0192c711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:01.566831 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674ba96-8f62-490d-bff0-872a0192c711 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:01.567167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674ba96-8f62-490d-bff0-872a0192c711 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:01.567447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674ba96-8f62-490d-bff0-872a0192c711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:01.576895 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 784/1563] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:01.577443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674ba96-8f62-490d-bff0-872a0192c711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:01.577794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483d672-ae38-4486-a881-c763c7b30ff3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:01.578925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674ba96-8f62-490d-bff0-872a0192c711 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:01.588590 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483d672-ae38-4486-a881-c763c7b30ff3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:01.589843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 780/1564] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:01.593651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989c5a2-70b9-4fc8-8725-7be4dda2c658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:01.594265 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989c5a2-70b9-4fc8-8725-7be4dda2c658 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:01.594544 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989c5a2-70b9-4fc8-8725-7be4dda2c658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:01.594839 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989c5a2-70b9-4fc8-8725-7be4dda2c658 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:01.595522 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989c5a2-70b9-4fc8-8725-7be4dda2c658 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:25:01.595829 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989c5a2-70b9-4fc8-8725-7be4dda2c658 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:01.598073 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989c5a2-70b9-4fc8-8725-7be4dda2c658 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:01.612241 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 785/1565] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:25:01.618044 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674ba96-8f62-490d-bff0-872a0192c711 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:01.628189 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674ba96-8f62-490d-bff0-872a0192c711 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:01.633553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3cf9b-2df5-410e-9541-d5194fc97f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:01.634165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3cf9b-2df5-410e-9541-d5194fc97f75 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:01.634443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3cf9b-2df5-410e-9541-d5194fc97f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:01.635091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3cf9b-2df5-410e-9541-d5194fc97f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:01.635375 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3cf9b-2df5-410e-9541-d5194fc97f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:01.637292 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3cf9b-2df5-410e-9541-d5194fc97f75 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:01.668608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 781/1566] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:01 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:01.673209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989c5a2-70b9-4fc8-8725-7be4dda2c658 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:01.683660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989c5a2-70b9-4fc8-8725-7be4dda2c658 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:01.772937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758246d-e746-47ae-87df-e6ad7cf50415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:01.773596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758246d-e746-47ae-87df-e6ad7cf50415 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:01.773882 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758246d-e746-47ae-87df-e6ad7cf50415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:01.774163 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758246d-e746-47ae-87df-e6ad7cf50415 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:01.774803 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758246d-e746-47ae-87df-e6ad7cf50415 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:01.776840 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758246d-e746-47ae-87df-e6ad7cf50415 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:01.806224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 786/1567] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:01 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:01.810240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3cf9b-2df5-410e-9541-d5194fc97f75 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:01.823907 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3cf9b-2df5-410e-9541-d5194fc97f75 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:01.835165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c47677-f369-45f0-9c9b-0a317613d60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:01.836051 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c47677-f369-45f0-9c9b-0a317613d60d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:01.836728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c47677-f369-45f0-9c9b-0a317613d60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:01.836728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c47677-f369-45f0-9c9b-0a317613d60d admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:01.837567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c47677-f369-45f0-9c9b-0a317613d60d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:01.839418 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c47677-f369-45f0-9c9b-0a317613d60d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:01.853840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 782/1568] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:01.857680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758246d-e746-47ae-87df-e6ad7cf50415 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:01.873280 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758246d-e746-47ae-87df-e6ad7cf50415 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:01.880349 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27410b10-749b-4828-9229-8ec0a57f6bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:01.881008 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27410b10-749b-4828-9229-8ec0a57f6bbc admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:01.881720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27410b10-749b-4828-9229-8ec0a57f6bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:01.881720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27410b10-749b-4828-9229-8ec0a57f6bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:01.882504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27410b10-749b-4828-9229-8ec0a57f6bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:01.884183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27410b10-749b-4828-9229-8ec0a57f6bbc admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:01.899575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 787/1569] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:01.904277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c47677-f369-45f0-9c9b-0a317613d60d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:01.917538 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c47677-f369-45f0-9c9b-0a317613d60d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:01.924737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512f26a-b873-49ad-8076-160942354417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:01.925586 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512f26a-b873-49ad-8076-160942354417 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:01.925928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512f26a-b873-49ad-8076-160942354417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:01.926148 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512f26a-b873-49ad-8076-160942354417 admin admin] PATH_INFO: `/v3/projects/f7374385eb294799984a042bd17f1ff0/users/4d9d4e9491ff470c903acfffb15caf33/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:01.932893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27410b10-749b-4828-9229-8ec0a57f6bbc admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:01.934709 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-27410b10-749b-4828-9229-8ec0a57f6bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:01.970571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27410b10-749b-4828-9229-8ec0a57f6bbc admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:01.977133 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:01.977788 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:01.978074 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:01.978348 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] PATH_INFO: `/v3/projects/3d4f62d58dd7415e88e4ff1430d8ae9b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:01.979545 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d4f62d58dd7415e88e4ff1430d8ae9b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:01.981090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512f26a-b873-49ad-8076-160942354417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7374385eb294799984a042bd17f1ff0, user_id=4d9d4e9491ff470c903acfffb15caf33, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:01.981486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:01.985118 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512f26a-b873-49ad-8076-160942354417 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:02.005742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 783/1570] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:01 2022] PUT /identity/v3/projects/f7374385eb294799984a042bd17f1ff0/users/4d9d4e9491ff470c903acfffb15caf33/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:25:02.018710 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650d6f5-896b-4b06-98b4-97a926222307 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:02.019912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650d6f5-896b-4b06-98b4-97a926222307 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:02.020329 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650d6f5-896b-4b06-98b4-97a926222307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:02.035856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:02.042557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d4f62d58dd7415e88e4ff1430d8ae9b', 'request_id': 'req-a7a3295c-d65a-489f-8868-eb57bd923abf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:02.053232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d4f62d58dd7415e88e4ff1430d8ae9b', 'request_id': 'req-a7a3295c-d65a-489f-8868-eb57bd923abf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:02.057176 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f650d6f5-896b-4b06-98b4-97a926222307 None None] MFA Rules not processed for user `4d9d4e9491ff470c903acfffb15caf33`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:02.058420 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f650d6f5-896b-4b06-98b4-97a926222307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:02.062449 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d4f62d58dd7415e88e4ff1430d8ae9b', 'request_id': 'req-a7a3295c-d65a-489f-8868-eb57bd923abf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:02.081480 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:02.100125 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] The token cache is being invalidate because project 3d4f62d58dd7415e88e4ff1430d8ae9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:02.100531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:02.101363 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:02.103419 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 788/1571] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:01 2022] DELETE /identity/v3/projects/3d4f62d58dd7415e88e4ff1430d8ae9b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:02.107271 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:02.109492 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:02.144784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 784/1572] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:25:02.149016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650d6f5-896b-4b06-98b4-97a926222307 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:02.150335 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f650d6f5-896b-4b06-98b4-97a926222307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:02.189593 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3295c-d65a-489f-8868-eb57bd923abf admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:02.195330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b030b-58b8-4dc6-aa9d-2ac32a388e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:02.196215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b030b-58b8-4dc6-aa9d-2ac32a388e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:02.196646 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b030b-58b8-4dc6-aa9d-2ac32a388e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:02.197057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b030b-58b8-4dc6-aa9d-2ac32a388e80 admin admin] PATH_INFO: `/v3/users/9e4489c178404a069215f6bc2b4b7d37` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:02.198344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b030b-58b8-4dc6-aa9d-2ac32a388e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e4489c178404a069215f6bc2b4b7d37)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:02.200778 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b030b-58b8-4dc6-aa9d-2ac32a388e80 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:02.227838 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650d6f5-896b-4b06-98b4-97a926222307 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:02.236662 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e5ec2-4991-4e53-ac1b-fe51e13373f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:02.237408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e5ec2-4991-4e53-ac1b-fe51e13373f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:02.237748 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e5ec2-4991-4e53-ac1b-fe51e13373f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:02.238194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e5ec2-4991-4e53-ac1b-fe51e13373f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:02.239268 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e5ec2-4991-4e53-ac1b-fe51e13373f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:02.241778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e5ec2-4991-4e53-ac1b-fe51e13373f8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:02.254994 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1d5b030b-58b8-4dc6-aa9d-2ac32a388e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e4489c178404a069215f6bc2b4b7d37'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:02.263742 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1d5b030b-58b8-4dc6-aa9d-2ac32a388e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e4489c178404a069215f6bc2b4b7d37'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:02.294397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 785/1573] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:02 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:25:02.294510 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1d5b030b-58b8-4dc6-aa9d-2ac32a388e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e4489c178404a069215f6bc2b4b7d37'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:02.294510 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1d5b030b-58b8-4dc6-aa9d-2ac32a388e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e4489c178404a069215f6bc2b4b7d37'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:02.294510 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1d5b030b-58b8-4dc6-aa9d-2ac32a388e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e4489c178404a069215f6bc2b4b7d37'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:02.295161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e5ec2-4991-4e53-ac1b-fe51e13373f8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:02.295161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3e1e5ec2-4991-4e53-ac1b-fe51e13373f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:02.295319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 789/1574] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:02 2022] DELETE /identity/v3/users/9e4489c178404a069215f6bc2b4b7d37 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:25:02.380145 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e5ec2-4991-4e53-ac1b-fe51e13373f8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:02.445473 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:02.465398 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be512233-5431-4851-bb93-ebe7b4a67161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:02.466710 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be512233-5431-4851-bb93-ebe7b4a67161 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:02.467366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be512233-5431-4851-bb93-ebe7b4a67161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:02.468363 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be512233-5431-4851-bb93-ebe7b4a67161 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:02.469738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be512233-5431-4851-bb93-ebe7b4a67161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:02.473242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be512233-5431-4851-bb93-ebe7b4a67161 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:02.512192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 786/1575] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:02 2022] POST /identity/v3/users => generated 294 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:02.514403 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b030b-58b8-4dc6-aa9d-2ac32a388e80 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:02.537081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b030b-58b8-4dc6-aa9d-2ac32a388e80 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:02.548847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60c207-4b76-4776-88ab-adc0bfa6db39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:02.549742 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60c207-4b76-4776-88ab-adc0bfa6db39 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:02.550741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60c207-4b76-4776-88ab-adc0bfa6db39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:02.551254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60c207-4b76-4776-88ab-adc0bfa6db39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:02.552014 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60c207-4b76-4776-88ab-adc0bfa6db39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:02.554502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60c207-4b76-4776-88ab-adc0bfa6db39 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:02.574407 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 790/1576] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:02.576693 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be512233-5431-4851-bb93-ebe7b4a67161 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:02.588047 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be512233-5431-4851-bb93-ebe7b4a67161 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:02.594028 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85bb0f-d7d2-4005-91bb-bdb0e095111b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:02.594715 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85bb0f-d7d2-4005-91bb-bdb0e095111b admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:02.596015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85bb0f-d7d2-4005-91bb-bdb0e095111b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:02.596331 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85bb0f-d7d2-4005-91bb-bdb0e095111b admin admin] PATH_INFO: `/v3/projects/79d967ef48b14dbaacdba8c2cc634e28/users/dcc27c3de0aa4b6da7325b2ecacd06aa/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:02.620053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85bb0f-d7d2-4005-91bb-bdb0e095111b admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d967ef48b14dbaacdba8c2cc634e28, user_id=dcc27c3de0aa4b6da7325b2ecacd06aa, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:02.624581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85bb0f-d7d2-4005-91bb-bdb0e095111b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:02.646640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 787/1577] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:02 2022] PUT /identity/v3/projects/79d967ef48b14dbaacdba8c2cc634e28/users/dcc27c3de0aa4b6da7325b2ecacd06aa/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:02.650710 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60c207-4b76-4776-88ab-adc0bfa6db39 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:02.662155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60c207-4b76-4776-88ab-adc0bfa6db39 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:02.699286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:02.732217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e25b1a-113c-426b-99f9-df8f3b09af41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:02.733112 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e25b1a-113c-426b-99f9-df8f3b09af41 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:02.733340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e25b1a-113c-426b-99f9-df8f3b09af41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:02.733669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e25b1a-113c-426b-99f9-df8f3b09af41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:02.734322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e25b1a-113c-426b-99f9-df8f3b09af41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:02.737295 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e25b1a-113c-426b-99f9-df8f3b09af41 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:02.754981 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 791/1578] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:02.756279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85bb0f-d7d2-4005-91bb-bdb0e095111b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:02.767222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85bb0f-d7d2-4005-91bb-bdb0e095111b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:02.772266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa2c93-4cc0-4486-b002-ceb84f6255e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:02.772966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa2c93-4cc0-4486-b002-ceb84f6255e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:02.773249 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa2c93-4cc0-4486-b002-ceb84f6255e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:02.773525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa2c93-4cc0-4486-b002-ceb84f6255e9 admin admin] PATH_INFO: `/v3/projects/79d967ef48b14dbaacdba8c2cc634e28/users/dcc27c3de0aa4b6da7325b2ecacd06aa/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:02.775561 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa2c93-4cc0-4486-b002-ceb84f6255e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d967ef48b14dbaacdba8c2cc634e28, user_id=dcc27c3de0aa4b6da7325b2ecacd06aa, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:02.778232 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa2c93-4cc0-4486-b002-ceb84f6255e9 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:02.795752 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 788/1579] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:02 2022] PUT /identity/v3/projects/79d967ef48b14dbaacdba8c2cc634e28/users/dcc27c3de0aa4b6da7325b2ecacd06aa/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:02.808846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59691af9-8236-4873-b9cd-8e855b7e8027 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:02.809195 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59691af9-8236-4873-b9cd-8e855b7e8027 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:02.809539 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59691af9-8236-4873-b9cd-8e855b7e8027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:02.841543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-59691af9-8236-4873-b9cd-8e855b7e8027 None None] MFA Rules not processed for user `dcc27c3de0aa4b6da7325b2ecacd06aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:02.842885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-59691af9-8236-4873-b9cd-8e855b7e8027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:02.922816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa2c93-4cc0-4486-b002-ceb84f6255e9 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:02.927713 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4faa2c93-4cc0-4486-b002-ceb84f6255e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:02.955528 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 792/1580] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:03.007232 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa2c93-4cc0-4486-b002-ceb84f6255e9 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:03.012229 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7eeec4-bd32-4194-82d3-20817eb6f17f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:03.013171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eeec4-bd32-4194-82d3-20817eb6f17f service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:03.013362 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eeec4-bd32-4194-82d3-20817eb6f17f service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:03.013682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eeec4-bd32-4194-82d3-20817eb6f17f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:03.015656 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6f7eeec4-bd32-4194-82d3-20817eb6f17f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:03.028434 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59691af9-8236-4873-b9cd-8e855b7e8027 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:03.032302 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-59691af9-8236-4873-b9cd-8e855b7e8027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:03.046116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59691af9-8236-4873-b9cd-8e855b7e8027 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:03.051349 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b8812-c85e-4305-8ffd-4599e1108910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:03.052159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b8812-c85e-4305-8ffd-4599e1108910 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:03.052470 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b8812-c85e-4305-8ffd-4599e1108910 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:03.052752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b8812-c85e-4305-8ffd-4599e1108910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:03.054771 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f07b8812-c85e-4305-8ffd-4599e1108910 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:03.113040 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7eeec4-bd32-4194-82d3-20817eb6f17f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:03.114951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7eeec4-bd32-4194-82d3-20817eb6f17f service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:03.132805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 789/1581] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:03.157811 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b8812-c85e-4305-8ffd-4599e1108910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:03.159434 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b8812-c85e-4305-8ffd-4599e1108910 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:03.173764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 793/1582] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:03.354116 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7eeec4-bd32-4194-82d3-20817eb6f17f service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:03.355576 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6f7eeec4-bd32-4194-82d3-20817eb6f17f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:03.462732 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:03.491167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7eeec4-bd32-4194-82d3-20817eb6f17f service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:03.514105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98893e-0585-4a57-ad08-2b200113477c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:03.516268 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98893e-0585-4a57-ad08-2b200113477c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:03.516577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98893e-0585-4a57-ad08-2b200113477c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:03.516858 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98893e-0585-4a57-ad08-2b200113477c admin admin] PATH_INFO: `/v3/projects/1be1e0ac2bbe4d83a7d22b291d6bd201` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:03.519988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98893e-0585-4a57-ad08-2b200113477c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1be1e0ac2bbe4d83a7d22b291d6bd201)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:03.523801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98893e-0585-4a57-ad08-2b200113477c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:03.660571 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cf98893e-0585-4a57-ad08-2b200113477c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1be1e0ac2bbe4d83a7d22b291d6bd201', 'request_id': 'req-cf98893e-0585-4a57-ad08-2b200113477c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:03.668473 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cf98893e-0585-4a57-ad08-2b200113477c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1be1e0ac2bbe4d83a7d22b291d6bd201', 'request_id': 'req-cf98893e-0585-4a57-ad08-2b200113477c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:03.685262 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cf98893e-0585-4a57-ad08-2b200113477c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1be1e0ac2bbe4d83a7d22b291d6bd201', 'request_id': 'req-cf98893e-0585-4a57-ad08-2b200113477c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:03.750532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cf98893e-0585-4a57-ad08-2b200113477c admin admin] The token cache is being invalidate because project 1be1e0ac2bbe4d83a7d22b291d6bd201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:03.753148 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cf98893e-0585-4a57-ad08-2b200113477c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:03.754316 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cf98893e-0585-4a57-ad08-2b200113477c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:03.760448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 790/1583] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:03 2022] DELETE /identity/v3/projects/1be1e0ac2bbe4d83a7d22b291d6bd201 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:03.765798 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb641d2-39af-4601-b841-e71845f35e23 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:03.766248 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb641d2-39af-4601-b841-e71845f35e23 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:03.766401 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb641d2-39af-4601-b841-e71845f35e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:03.822442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2eb641d2-39af-4601-b841-e71845f35e23 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:03.829786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2eb641d2-39af-4601-b841-e71845f35e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:03.914582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:03.925673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 794/1584] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:03.934524 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27176c62-c15e-4e89-ae1c-696f71cdcb48 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:03.935013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27176c62-c15e-4e89-ae1c-696f71cdcb48 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:03.935120 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27176c62-c15e-4e89-ae1c-696f71cdcb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:03.963012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-27176c62-c15e-4e89-ae1c-696f71cdcb48 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:03.964458 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-27176c62-c15e-4e89-ae1c-696f71cdcb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:03.973340 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 791/1585] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:03.977547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb641d2-39af-4601-b841-e71845f35e23 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:03.990102 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb641d2-39af-4601-b841-e71845f35e23 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:03.995749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9be2f6-c6e3-4a11-905a-a502c965cf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:03.996810 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9be2f6-c6e3-4a11-905a-a502c965cf11 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:03.996810 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9be2f6-c6e3-4a11-905a-a502c965cf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:03.997264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9be2f6-c6e3-4a11-905a-a502c965cf11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:03.998466 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9be2f6-c6e3-4a11-905a-a502c965cf11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:25:03.998884 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9be2f6-c6e3-4a11-905a-a502c965cf11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:04.000664 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9be2f6-c6e3-4a11-905a-a502c965cf11 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:04.017166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 795/1586] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:25:04.022163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27176c62-c15e-4e89-ae1c-696f71cdcb48 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:04.036944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27176c62-c15e-4e89-ae1c-696f71cdcb48 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:04.043567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb7888-e921-43b7-80a4-c84e98b543d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:04.044216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb7888-e921-43b7-80a4-c84e98b543d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:04.044495 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb7888-e921-43b7-80a4-c84e98b543d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:04.044793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb7888-e921-43b7-80a4-c84e98b543d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:04.045610 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb7888-e921-43b7-80a4-c84e98b543d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:04.047952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb7888-e921-43b7-80a4-c84e98b543d4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:04.109220 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9be2f6-c6e3-4a11-905a-a502c965cf11 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:04.118621 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 792/1587] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:04 2022] POST /identity/v3/projects => generated 357 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:04.124957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9be2f6-c6e3-4a11-905a-a502c965cf11 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:04.248631 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ceabf-1666-457e-b575-f417ac59e823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:04.249610 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ceabf-1666-457e-b575-f417ac59e823 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:04.250094 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ceabf-1666-457e-b575-f417ac59e823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:04.250561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ceabf-1666-457e-b575-f417ac59e823 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:04.251481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ceabf-1666-457e-b575-f417ac59e823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:04.254979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ceabf-1666-457e-b575-f417ac59e823 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:04.282916 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 796/1588] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:04 2022] POST /identity/v3/users => generated 279 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:04.287522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb7888-e921-43b7-80a4-c84e98b543d4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:04.295186 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ceabf-1666-457e-b575-f417ac59e823 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:04.297190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8e5ceabf-1666-457e-b575-f417ac59e823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:04.300954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb7888-e921-43b7-80a4-c84e98b543d4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:04.308230 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46407a9a-a6df-431a-add6-d72ed9193738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:04.309275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46407a9a-a6df-431a-add6-d72ed9193738 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:04.310031 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46407a9a-a6df-431a-add6-d72ed9193738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:04.310239 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46407a9a-a6df-431a-add6-d72ed9193738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:04.311099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46407a9a-a6df-431a-add6-d72ed9193738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:04.313245 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46407a9a-a6df-431a-add6-d72ed9193738 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:04.314866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ceabf-1666-457e-b575-f417ac59e823 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:04.322021 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060e485-7991-442a-bd97-3abd3ac31c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:04.323177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060e485-7991-442a-bd97-3abd3ac31c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:04.323602 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060e485-7991-442a-bd97-3abd3ac31c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:04.324170 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060e485-7991-442a-bd97-3abd3ac31c33 admin admin] PATH_INFO: `/v3/users/16b3f322fd3f4709800c6e7f850d4a84` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:04.325658 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060e485-7991-442a-bd97-3abd3ac31c33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16b3f322fd3f4709800c6e7f850d4a84)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:04.327870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060e485-7991-442a-bd97-3abd3ac31c33 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:04.328117 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 793/1589] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:04.331951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46407a9a-a6df-431a-add6-d72ed9193738 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:04.344390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46407a9a-a6df-431a-add6-d72ed9193738 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:04.351954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496b646-fd3a-4439-a00f-5a3a90347e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:04.352313 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496b646-fd3a-4439-a00f-5a3a90347e2b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:04.352705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496b646-fd3a-4439-a00f-5a3a90347e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:04.353344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496b646-fd3a-4439-a00f-5a3a90347e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:04.354332 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496b646-fd3a-4439-a00f-5a3a90347e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:04.356221 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496b646-fd3a-4439-a00f-5a3a90347e2b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:04.371151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 794/1590] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:04.373360 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496b646-fd3a-4439-a00f-5a3a90347e2b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:04.386717 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496b646-fd3a-4439-a00f-5a3a90347e2b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:04.392007 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c62435-06f7-4ede-b8c6-8ba1941b1450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:04.392755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c62435-06f7-4ede-b8c6-8ba1941b1450 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:04.393134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c62435-06f7-4ede-b8c6-8ba1941b1450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:04.393639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c62435-06f7-4ede-b8c6-8ba1941b1450 admin admin] PATH_INFO: `/v3/projects/052676dc098d42d6913991a727fa2e86/users/9b022072875848e78f976360efbbf90c/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:04.398849 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5060e485-7991-442a-bd97-3abd3ac31c33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16b3f322fd3f4709800c6e7f850d4a84'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:04.407881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5060e485-7991-442a-bd97-3abd3ac31c33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16b3f322fd3f4709800c6e7f850d4a84'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:04.412257 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c62435-06f7-4ede-b8c6-8ba1941b1450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=052676dc098d42d6913991a727fa2e86, user_id=9b022072875848e78f976360efbbf90c, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:04.415252 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c62435-06f7-4ede-b8c6-8ba1941b1450 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:04.424263 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5060e485-7991-442a-bd97-3abd3ac31c33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16b3f322fd3f4709800c6e7f850d4a84'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:04.430124 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 795/1591] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:04 2022] PUT /identity/v3/projects/052676dc098d42d6913991a727fa2e86/users/9b022072875848e78f976360efbbf90c/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:25:04.442851 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e15b-85b9-4c7b-b120-fdb758536038 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:04.443345 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5060e485-7991-442a-bd97-3abd3ac31c33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16b3f322fd3f4709800c6e7f850d4a84'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:04.443643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e15b-85b9-4c7b-b120-fdb758536038 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:04.444421 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e15b-85b9-4c7b-b120-fdb758536038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:04.444537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5060e485-7991-442a-bd97-3abd3ac31c33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16b3f322fd3f4709800c6e7f850d4a84'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:04.450959 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 797/1592] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:04 2022] DELETE /identity/v3/users/16b3f322fd3f4709800c6e7f850d4a84 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:04.455615 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb350f28-4544-454a-8311-c49a73b335e4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:04.455964 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb350f28-4544-454a-8311-c49a73b335e4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:04.456962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb350f28-4544-454a-8311-c49a73b335e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:04.480054 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-cb350f28-4544-454a-8311-c49a73b335e4 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:04.481374 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cb350f28-4544-454a-8311-c49a73b335e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:04.489328 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-0398e15b-85b9-4c7b-b120-fdb758536038 None None] MFA Rules not processed for user `9b022072875848e78f976360efbbf90c`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:04.491908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0398e15b-85b9-4c7b-b120-fdb758536038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:04.543455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:04.575206 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 798/1593] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:04.581687 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb350f28-4544-454a-8311-c49a73b335e4 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:04.583120 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cb350f28-4544-454a-8311-c49a73b335e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:04.583256 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:04.607124 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 796/1594] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:04 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:25:04.647947 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398e15b-85b9-4c7b-b120-fdb758536038 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:04.649872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0398e15b-85b9-4c7b-b120-fdb758536038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:04.679397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb350f28-4544-454a-8311-c49a73b335e4 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:04.683790 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83a235-7e04-4ffc-82a1-4f98e87423ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:04.684716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83a235-7e04-4ffc-82a1-4f98e87423ac service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:04.685205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83a235-7e04-4ffc-82a1-4f98e87423ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:04.685997 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83a235-7e04-4ffc-82a1-4f98e87423ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:04.697871 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83a235-7e04-4ffc-82a1-4f98e87423ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:04.699431 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83a235-7e04-4ffc-82a1-4f98e87423ac service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:04.711017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 799/1595] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:04.712158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84e687-e0ac-4747-9a95-76086806a626 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:04.712470 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84e687-e0ac-4747-9a95-76086806a626 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:04.712703 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84e687-e0ac-4747-9a95-76086806a626 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:04.714166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 800/1596] 10.222.0.18 () {54 vars in 921 bytes} [Fri Oct 14 02:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:25:04.717848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398e15b-85b9-4c7b-b120-fdb758536038 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:04.722734 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728ac04-85f6-4a44-b915-d53fc5ba6a7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:04.723392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728ac04-85f6-4a44-b915-d53fc5ba6a7c service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:04.723675 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728ac04-85f6-4a44-b915-d53fc5ba6a7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:04.723956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728ac04-85f6-4a44-b915-d53fc5ba6a7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:04.744251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6d854-2aad-4bd6-8eb1-3b3e634cd372 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:04.744597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6d854-2aad-4bd6-8eb1-3b3e634cd372 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:04.744914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6d854-2aad-4bd6-8eb1-3b3e634cd372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:04.753645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728ac04-85f6-4a44-b915-d53fc5ba6a7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:04.755514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728ac04-85f6-4a44-b915-d53fc5ba6a7c service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:04.771107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 797/1597] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:04.773886 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ffa6d854-2aad-4bd6-8eb1-3b3e634cd372 None None] MFA Rules not processed for user `de13be55b4034477baf4c029c1a79757`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:04.775196 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ffa6d854-2aad-4bd6-8eb1-3b3e634cd372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:04.787806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728ac04-85f6-4a44-b915-d53fc5ba6a7c service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:04.789615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c728ac04-85f6-4a44-b915-d53fc5ba6a7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:04.808736 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728ac04-85f6-4a44-b915-d53fc5ba6a7c service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:04.816873 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803e648-7d1d-4506-9fd4-a90cfaeb756c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:04.817947 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803e648-7d1d-4506-9fd4-a90cfaeb756c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:04.818445 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803e648-7d1d-4506-9fd4-a90cfaeb756c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:04.818958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803e648-7d1d-4506-9fd4-a90cfaeb756c admin admin] PATH_INFO: `/v3/users/9b022072875848e78f976360efbbf90c` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:04.820930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803e648-7d1d-4506-9fd4-a90cfaeb756c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b022072875848e78f976360efbbf90c)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:04.823568 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803e648-7d1d-4506-9fd4-a90cfaeb756c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:04.921351 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 801/1598] 10.222.0.18 () {58 vars in 1018 bytes} [Fri Oct 14 02:25:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:04.959180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7803e648-7d1d-4506-9fd4-a90cfaeb756c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b022072875848e78f976360efbbf90c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:04.968290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7803e648-7d1d-4506-9fd4-a90cfaeb756c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b022072875848e78f976360efbbf90c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:04.969166 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7803e648-7d1d-4506-9fd4-a90cfaeb756c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b022072875848e78f976360efbbf90c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:04.985252 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7803e648-7d1d-4506-9fd4-a90cfaeb756c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b022072875848e78f976360efbbf90c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:04.993986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7803e648-7d1d-4506-9fd4-a90cfaeb756c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b022072875848e78f976360efbbf90c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.003372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 798/1599] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:04 2022] DELETE /identity/v3/users/9b022072875848e78f976360efbbf90c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:05.008034 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60935f0f-0a73-4658-8359-8852217d5611 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:05.008484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60935f0f-0a73-4658-8359-8852217d5611 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:05.008727 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60935f0f-0a73-4658-8359-8852217d5611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:05.041139 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-60935f0f-0a73-4658-8359-8852217d5611 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:05.042504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-60935f0f-0a73-4658-8359-8852217d5611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:05.142365 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 802/1600] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:05.152509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803e648-7d1d-4506-9fd4-a90cfaeb756c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:05.154314 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7803e648-7d1d-4506-9fd4-a90cfaeb756c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:05.159049 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60935f0f-0a73-4658-8359-8852217d5611 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:05.169225 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-60935f0f-0a73-4658-8359-8852217d5611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:05.216058 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:05.250290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803e648-7d1d-4506-9fd4-a90cfaeb756c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:05.256622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cb2e2-486e-4a2c-b33f-18648658b447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:05.257732 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cb2e2-486e-4a2c-b33f-18648658b447 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:05.258278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cb2e2-486e-4a2c-b33f-18648658b447 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:05.258766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cb2e2-486e-4a2c-b33f-18648658b447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:05.274608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cb2e2-486e-4a2c-b33f-18648658b447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:05.276758 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cb2e2-486e-4a2c-b33f-18648658b447 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:05.286451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60935f0f-0a73-4658-8359-8852217d5611 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:05.288570 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 799/1601] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:05.296746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cb2e2-486e-4a2c-b33f-18648658b447 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:05.298365 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b66cb2e2-486e-4a2c-b33f-18648658b447 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:05.304321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c9313-c67a-4274-99de-ff185c930300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:05.304949 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c9313-c67a-4274-99de-ff185c930300 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:05.305237 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c9313-c67a-4274-99de-ff185c930300 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:05.305514 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c9313-c67a-4274-99de-ff185c930300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:05.312105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-de3c9313-c67a-4274-99de-ff185c930300 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:05.318658 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cb2e2-486e-4a2c-b33f-18648658b447 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:05.324692 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:05.325352 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:05.325631 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:05.325904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] PATH_INFO: `/v3/projects/392db12b4b4449f19aeaa8e477ce0b1d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:05.327306 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=392db12b4b4449f19aeaa8e477ce0b1d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:05.333731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c9313-c67a-4274-99de-ff185c930300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:05.335937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c9313-c67a-4274-99de-ff185c930300 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:05.336203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:05.347567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:05.360435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 803/1602] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:05.434980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '392db12b4b4449f19aeaa8e477ce0b1d', 'request_id': 'req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.443977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '392db12b4b4449f19aeaa8e477ce0b1d', 'request_id': 'req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.452375 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '392db12b4b4449f19aeaa8e477ce0b1d', 'request_id': 'req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.473489 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] The token cache is being invalidate because project 392db12b4b4449f19aeaa8e477ce0b1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:05.473787 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.474508 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.476311 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 800/1603] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:05 2022] DELETE /identity/v3/projects/392db12b4b4449f19aeaa8e477ce0b1d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:05.480445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c9313-c67a-4274-99de-ff185c930300 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:05.493597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c9313-c67a-4274-99de-ff185c930300 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:05.571235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8af954-73aa-4f10-85f5-c7d3ebbc6572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:05.571873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8af954-73aa-4f10-85f5-c7d3ebbc6572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:05.572153 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8af954-73aa-4f10-85f5-c7d3ebbc6572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:05.572429 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8af954-73aa-4f10-85f5-c7d3ebbc6572 admin admin] PATH_INFO: `/v3/users/5b98c9dce6be48129aadc64bbfae57af` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:05.573560 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8af954-73aa-4f10-85f5-c7d3ebbc6572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b98c9dce6be48129aadc64bbfae57af)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:05.575542 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8af954-73aa-4f10-85f5-c7d3ebbc6572 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:05.608372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:05.609995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:05.625288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa15b1e-c33b-4da2-bd1d-6cc7175511c7 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:05.633377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0a8af954-73aa-4f10-85f5-c7d3ebbc6572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b98c9dce6be48129aadc64bbfae57af'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.642479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0a8af954-73aa-4f10-85f5-c7d3ebbc6572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b98c9dce6be48129aadc64bbfae57af'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.645873 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec0720-7336-4f50-b209-aec3151f3dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:05.646541 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec0720-7336-4f50-b209-aec3151f3dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:05.646824 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec0720-7336-4f50-b209-aec3151f3dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:05.647118 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec0720-7336-4f50-b209-aec3151f3dda admin admin] PATH_INFO: `/v3/projects/052676dc098d42d6913991a727fa2e86` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:05.649119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec0720-7336-4f50-b209-aec3151f3dda admin admin] RBAC: Authorizing `identity:delete_project(project_id=052676dc098d42d6913991a727fa2e86)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:05.651161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec0720-7336-4f50-b209-aec3151f3dda admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:05.658503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0a8af954-73aa-4f10-85f5-c7d3ebbc6572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b98c9dce6be48129aadc64bbfae57af'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.667406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0a8af954-73aa-4f10-85f5-c7d3ebbc6572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b98c9dce6be48129aadc64bbfae57af'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.668257 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0a8af954-73aa-4f10-85f5-c7d3ebbc6572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b98c9dce6be48129aadc64bbfae57af'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.679833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 804/1604] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:05 2022] DELETE /identity/v3/users/5b98c9dce6be48129aadc64bbfae57af => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:05.709836 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a9ec0720-7336-4f50-b209-aec3151f3dda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '052676dc098d42d6913991a727fa2e86', 'request_id': 'req-a9ec0720-7336-4f50-b209-aec3151f3dda'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.717007 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a9ec0720-7336-4f50-b209-aec3151f3dda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '052676dc098d42d6913991a727fa2e86', 'request_id': 'req-a9ec0720-7336-4f50-b209-aec3151f3dda'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.727535 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a9ec0720-7336-4f50-b209-aec3151f3dda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '052676dc098d42d6913991a727fa2e86', 'request_id': 'req-a9ec0720-7336-4f50-b209-aec3151f3dda'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.746919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a9ec0720-7336-4f50-b209-aec3151f3dda admin admin] The token cache is being invalidate because project 052676dc098d42d6913991a727fa2e86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:05.747278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a9ec0720-7336-4f50-b209-aec3151f3dda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.748291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a9ec0720-7336-4f50-b209-aec3151f3dda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:05.750440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 801/1605] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:05 2022] DELETE /identity/v3/projects/052676dc098d42d6913991a727fa2e86 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:25:05.755982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85afc6-a93d-40a2-916e-f534471786a6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:05.756344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85afc6-a93d-40a2-916e-f534471786a6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:05.756959 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85afc6-a93d-40a2-916e-f534471786a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:05.780595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6e85afc6-a93d-40a2-916e-f534471786a6 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:05.781832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6e85afc6-a93d-40a2-916e-f534471786a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:05.805706 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:05.854993 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 805/1606] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:05.863837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fa9d4-916b-438b-ba8e-8dcf158a018a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:05.864347 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fa9d4-916b-438b-ba8e-8dcf158a018a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:05.864627 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fa9d4-916b-438b-ba8e-8dcf158a018a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:05.885607 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b40fa9d4-916b-438b-ba8e-8dcf158a018a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:05.886820 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b40fa9d4-916b-438b-ba8e-8dcf158a018a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:05.895424 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 802/1607] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:05.899743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85afc6-a93d-40a2-916e-f534471786a6 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:05.910166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85afc6-a93d-40a2-916e-f534471786a6 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:05.914705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0f530-9c8a-45fa-9100-e7dd04e7e614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:05.915341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0f530-9c8a-45fa-9100-e7dd04e7e614 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:05.915616 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0f530-9c8a-45fa-9100-e7dd04e7e614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:05.915940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0f530-9c8a-45fa-9100-e7dd04e7e614 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:05.916601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0f530-9c8a-45fa-9100-e7dd04e7e614 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:25:05.916925 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0f530-9c8a-45fa-9100-e7dd04e7e614 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:05.918551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0f530-9c8a-45fa-9100-e7dd04e7e614 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:05.940593 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 806/1608] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:25:05.942164 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fa9d4-916b-438b-ba8e-8dcf158a018a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:05.952076 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fa9d4-916b-438b-ba8e-8dcf158a018a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:05.957665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40f16f-5e4a-49f1-8590-9a54c2cf781b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:05.958307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40f16f-5e4a-49f1-8590-9a54c2cf781b admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:05.958588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40f16f-5e4a-49f1-8590-9a54c2cf781b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:05.958943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40f16f-5e4a-49f1-8590-9a54c2cf781b admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:05.959619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40f16f-5e4a-49f1-8590-9a54c2cf781b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:05.961595 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40f16f-5e4a-49f1-8590-9a54c2cf781b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:05.992410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 803/1609] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:05 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:05.996559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0f530-9c8a-45fa-9100-e7dd04e7e614 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:06.009188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0f530-9c8a-45fa-9100-e7dd04e7e614 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:06.110046 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adb67a-e811-4cde-a401-16684447328e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:06.110717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adb67a-e811-4cde-a401-16684447328e admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.111145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adb67a-e811-4cde-a401-16684447328e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.111568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adb67a-e811-4cde-a401-16684447328e admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.112150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8adb67a-e811-4cde-a401-16684447328e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:06.115598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8adb67a-e811-4cde-a401-16684447328e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:06.176146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 807/1610] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:05 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:25:06.180352 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40f16f-5e4a-49f1-8590-9a54c2cf781b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:06.199647 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40f16f-5e4a-49f1-8590-9a54c2cf781b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:06.216876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04cbdd-6250-4185-aac0-5accabaebba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:06.217711 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04cbdd-6250-4185-aac0-5accabaebba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.218056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04cbdd-6250-4185-aac0-5accabaebba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.218392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04cbdd-6250-4185-aac0-5accabaebba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.219249 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04cbdd-6250-4185-aac0-5accabaebba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:06.221048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04cbdd-6250-4185-aac0-5accabaebba6 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:06.242131 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adb67a-e811-4cde-a401-16684447328e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:06.244236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a8adb67a-e811-4cde-a401-16684447328e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:06.264117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adb67a-e811-4cde-a401-16684447328e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:06.275912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 804/1611] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:06.279741 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04cbdd-6250-4185-aac0-5accabaebba6 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:06.286846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c3060-07c2-483f-bb77-d16571c9f230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:06.287587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c3060-07c2-483f-bb77-d16571c9f230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.288109 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c3060-07c2-483f-bb77-d16571c9f230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.288259 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c3060-07c2-483f-bb77-d16571c9f230 admin admin] PATH_INFO: `/v3/projects/2cebd13aeeed4462b4a6eb2f76c82e4f` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.289452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c3060-07c2-483f-bb77-d16571c9f230 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cebd13aeeed4462b4a6eb2f76c82e4f)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:06.291561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c3060-07c2-483f-bb77-d16571c9f230 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:06.294437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04cbdd-6250-4185-aac0-5accabaebba6 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:06.299609 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbf5f2-009c-4c36-87dd-97f5f23983a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:06.300551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbf5f2-009c-4c36-87dd-97f5f23983a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.301801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbf5f2-009c-4c36-87dd-97f5f23983a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.302852 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbf5f2-009c-4c36-87dd-97f5f23983a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.303749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cbf5f2-009c-4c36-87dd-97f5f23983a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:06.305508 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cbf5f2-009c-4c36-87dd-97f5f23983a4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:06.321298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 805/1612] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:06.322357 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbf5f2-009c-4c36-87dd-97f5f23983a4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:06.333908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbf5f2-009c-4c36-87dd-97f5f23983a4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:06.342059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2501ef-24ea-41e8-884f-b7b225ebeff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:06.343285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2501ef-24ea-41e8-884f-b7b225ebeff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.343715 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2501ef-24ea-41e8-884f-b7b225ebeff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.345527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2501ef-24ea-41e8-884f-b7b225ebeff0 admin admin] PATH_INFO: `/v3/projects/8d19ceadff46434ea8435f9cb13c2700/users/2706c08285894193977e6dd2ed14afc4/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.365624 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2501ef-24ea-41e8-884f-b7b225ebeff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d19ceadff46434ea8435f9cb13c2700, user_id=2706c08285894193977e6dd2ed14afc4, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:06.367168 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6e4c3060-07c2-483f-bb77-d16571c9f230 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cebd13aeeed4462b4a6eb2f76c82e4f', 'request_id': 'req-6e4c3060-07c2-483f-bb77-d16571c9f230'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:06.368472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2501ef-24ea-41e8-884f-b7b225ebeff0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:06.374304 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6e4c3060-07c2-483f-bb77-d16571c9f230 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cebd13aeeed4462b4a6eb2f76c82e4f', 'request_id': 'req-6e4c3060-07c2-483f-bb77-d16571c9f230'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:06.381987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6e4c3060-07c2-483f-bb77-d16571c9f230 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cebd13aeeed4462b4a6eb2f76c82e4f', 'request_id': 'req-6e4c3060-07c2-483f-bb77-d16571c9f230'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:06.390908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 806/1613] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:06 2022] PUT /identity/v3/projects/8d19ceadff46434ea8435f9cb13c2700/users/2706c08285894193977e6dd2ed14afc4/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:06.401866 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216252a-8e33-44bd-a627-df5f2c1e55cb None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.402154 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216252a-8e33-44bd-a627-df5f2c1e55cb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.402500 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216252a-8e33-44bd-a627-df5f2c1e55cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.407285 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6e4c3060-07c2-483f-bb77-d16571c9f230 admin admin] The token cache is being invalidate because project 2cebd13aeeed4462b4a6eb2f76c82e4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:06.407691 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6e4c3060-07c2-483f-bb77-d16571c9f230 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:06.408380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6e4c3060-07c2-483f-bb77-d16571c9f230 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:06.410615 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 808/1614] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:06 2022] DELETE /identity/v3/projects/2cebd13aeeed4462b4a6eb2f76c82e4f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:06.411553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:06.415547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca653fd4-35e6-4ebd-8f5c-3122279f7830 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.415889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca653fd4-35e6-4ebd-8f5c-3122279f7830 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.416210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca653fd4-35e6-4ebd-8f5c-3122279f7830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.439700 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4216252a-8e33-44bd-a627-df5f2c1e55cb None None] MFA Rules not processed for user `2706c08285894193977e6dd2ed14afc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:06.439905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ca653fd4-35e6-4ebd-8f5c-3122279f7830 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:06.441845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ca653fd4-35e6-4ebd-8f5c-3122279f7830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:06.442003 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4216252a-8e33-44bd-a627-df5f2c1e55cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:06.467854 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:06.533558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 809/1615] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:25:06.540602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 807/1616] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:06.545447 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b8614-bb38-4ec4-b7da-d0eea0319452 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.545742 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b8614-bb38-4ec4-b7da-d0eea0319452 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.546020 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b8614-bb38-4ec4-b7da-d0eea0319452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.556854 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216252a-8e33-44bd-a627-df5f2c1e55cb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:06.559763 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4216252a-8e33-44bd-a627-df5f2c1e55cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:06.569209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c30b8614-bb38-4ec4-b7da-d0eea0319452 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:06.570768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c30b8614-bb38-4ec4-b7da-d0eea0319452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:06.582509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 810/1617] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:06.583501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b8614-bb38-4ec4-b7da-d0eea0319452 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:06.598216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b8614-bb38-4ec4-b7da-d0eea0319452 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:06.603600 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e99e5-6961-423d-bab4-c0b29e90a60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:06.604272 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e99e5-6961-423d-bab4-c0b29e90a60c admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.604545 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e99e5-6961-423d-bab4-c0b29e90a60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.604856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e99e5-6961-423d-bab4-c0b29e90a60c admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.605517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e99e5-6961-423d-bab4-c0b29e90a60c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:25:06.605818 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e99e5-6961-423d-bab4-c0b29e90a60c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:06.607557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e99e5-6961-423d-bab4-c0b29e90a60c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:06.621106 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 811/1618] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:25:06.625479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e99e5-6961-423d-bab4-c0b29e90a60c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:06.639405 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216252a-8e33-44bd-a627-df5f2c1e55cb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:06.640462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e99e5-6961-423d-bab4-c0b29e90a60c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:06.643286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c5873-b51c-4342-b80b-0a7cc1d772c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:06.644737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c5873-b51c-4342-b80b-0a7cc1d772c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.644737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c5873-b51c-4342-b80b-0a7cc1d772c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.644992 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c5873-b51c-4342-b80b-0a7cc1d772c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.645724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02890c-aca1-4c74-9560-5ea6a73ba29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:06.646339 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02890c-aca1-4c74-9560-5ea6a73ba29f admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.646624 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02890c-aca1-4c74-9560-5ea6a73ba29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.646991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02890c-aca1-4c74-9560-5ea6a73ba29f admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.647566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02890c-aca1-4c74-9560-5ea6a73ba29f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:06.649473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02890c-aca1-4c74-9560-5ea6a73ba29f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:06.654537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c5873-b51c-4342-b80b-0a7cc1d772c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:06.656046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c5873-b51c-4342-b80b-0a7cc1d772c1 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:06.667955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 808/1619] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:06.692972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 812/1620] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:06 2022] POST /identity/v3/projects => generated 369 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:06.697133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c5873-b51c-4342-b80b-0a7cc1d772c1 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:06.713201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c5873-b51c-4342-b80b-0a7cc1d772c1 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:06.775149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4bf98-7526-4262-becf-c61f72e1a0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:06.775773 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4bf98-7526-4262-becf-c61f72e1a0ba admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.776059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4bf98-7526-4262-becf-c61f72e1a0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.776332 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4bf98-7526-4262-becf-c61f72e1a0ba admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.776996 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4bf98-7526-4262-becf-c61f72e1a0ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:06.779041 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4bf98-7526-4262-becf-c61f72e1a0ba admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:06.802598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 809/1621] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:06 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:06.806766 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02890c-aca1-4c74-9560-5ea6a73ba29f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:06.818095 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02890c-aca1-4c74-9560-5ea6a73ba29f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:06.824152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78973167-890f-4e81-a0d1-c48c16661edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:06.824780 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78973167-890f-4e81-a0d1-c48c16661edf admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.825076 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78973167-890f-4e81-a0d1-c48c16661edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.825362 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78973167-890f-4e81-a0d1-c48c16661edf admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.826020 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78973167-890f-4e81-a0d1-c48c16661edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:06.827757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78973167-890f-4e81-a0d1-c48c16661edf admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:06.839035 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 813/1622] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:25:06.844215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4bf98-7526-4262-becf-c61f72e1a0ba admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:06.855328 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4bf98-7526-4262-becf-c61f72e1a0ba admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:06.860247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510796da-aced-4d03-ad97-271f4fc75196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:06.860878 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510796da-aced-4d03-ad97-271f4fc75196 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.861158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510796da-aced-4d03-ad97-271f4fc75196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.861450 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510796da-aced-4d03-ad97-271f4fc75196 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.862675 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510796da-aced-4d03-ad97-271f4fc75196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:06.864762 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510796da-aced-4d03-ad97-271f4fc75196 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:06.875575 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 810/1623] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:25:06.880011 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78973167-890f-4e81-a0d1-c48c16661edf admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:06.890676 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78973167-890f-4e81-a0d1-c48c16661edf admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:06.895902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe055b-9ed9-44b3-97e4-2afbb06bcb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:06.896590 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe055b-9ed9-44b3-97e4-2afbb06bcb06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.896892 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe055b-9ed9-44b3-97e4-2afbb06bcb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.897197 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe055b-9ed9-44b3-97e4-2afbb06bcb06 admin admin] PATH_INFO: `/v3/projects/5e5d76e3b1fe4f59acf33cfaaa1c02f8/users/53f68132b44649438a3977d02792702b/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.913600 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe055b-9ed9-44b3-97e4-2afbb06bcb06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e5d76e3b1fe4f59acf33cfaaa1c02f8, user_id=53f68132b44649438a3977d02792702b, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:06.916315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe055b-9ed9-44b3-97e4-2afbb06bcb06 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:06.930679 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 814/1624] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:06 2022] PUT /identity/v3/projects/5e5d76e3b1fe4f59acf33cfaaa1c02f8/users/53f68132b44649438a3977d02792702b/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:06.941601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5241267-ba58-497c-a3cb-fd5ef740937d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:06.941896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5241267-ba58-497c-a3cb-fd5ef740937d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:06.942245 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5241267-ba58-497c-a3cb-fd5ef740937d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:06.972166 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:06.978304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c5241267-ba58-497c-a3cb-fd5ef740937d None None] MFA Rules not processed for user `53f68132b44649438a3977d02792702b`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:06.980531 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c5241267-ba58-497c-a3cb-fd5ef740937d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:07.070885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 811/1625] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:07.075678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe055b-9ed9-44b3-97e4-2afbb06bcb06 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:07.181194 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe055b-9ed9-44b3-97e4-2afbb06bcb06 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:07.188095 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a534b13-bfd7-4f71-b0f4-c550c8b797a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:07.188953 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a534b13-bfd7-4f71-b0f4-c550c8b797a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:07.189378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a534b13-bfd7-4f71-b0f4-c550c8b797a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:07.189903 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a534b13-bfd7-4f71-b0f4-c550c8b797a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:07.191268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a534b13-bfd7-4f71-b0f4-c550c8b797a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:07.193672 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a534b13-bfd7-4f71-b0f4-c550c8b797a9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:07.231721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 815/1626] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:07 2022] POST /identity/v3/projects => generated 367 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:07.231927 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5241267-ba58-497c-a3cb-fd5ef740937d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:07.247011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5241267-ba58-497c-a3cb-fd5ef740937d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:07.347932 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930c821-a6b7-4476-8320-829c7ace5d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:07.350289 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930c821-a6b7-4476-8320-829c7ace5d0f admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:07.350763 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930c821-a6b7-4476-8320-829c7ace5d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:07.351266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930c821-a6b7-4476-8320-829c7ace5d0f admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:07.352089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930c821-a6b7-4476-8320-829c7ace5d0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:07.354475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930c821-a6b7-4476-8320-829c7ace5d0f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:07.390413 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a534b13-bfd7-4f71-b0f4-c550c8b797a9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:07.401490 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 812/1627] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:07 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:25:07.404563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a534b13-bfd7-4f71-b0f4-c550c8b797a9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:07.416162 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d6271-a7ff-41c4-a8f2-aa7924382d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:07.417201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d6271-a7ff-41c4-a8f2-aa7924382d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:07.417642 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d6271-a7ff-41c4-a8f2-aa7924382d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:07.418138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d6271-a7ff-41c4-a8f2-aa7924382d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:07.419021 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d6271-a7ff-41c4-a8f2-aa7924382d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:07.421471 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d6271-a7ff-41c4-a8f2-aa7924382d24 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:07.439274 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 816/1628] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:07.440962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930c821-a6b7-4476-8320-829c7ace5d0f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:07.454396 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930c821-a6b7-4476-8320-829c7ace5d0f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:07.460870 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae743a46-8f03-4739-af11-58f915c35cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:07.461602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae743a46-8f03-4739-af11-58f915c35cc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:07.461886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae743a46-8f03-4739-af11-58f915c35cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:07.462216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae743a46-8f03-4739-af11-58f915c35cc8 admin admin] PATH_INFO: `/v3/projects/98b48685bfc94524a3651762260da0c3/users/2b98b3146a4649e28ae48a403062f2d2/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:07.498467 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae743a46-8f03-4739-af11-58f915c35cc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b48685bfc94524a3651762260da0c3, user_id=2b98b3146a4649e28ae48a403062f2d2, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:07.502747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae743a46-8f03-4739-af11-58f915c35cc8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:07.550624 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d6271-a7ff-41c4-a8f2-aa7924382d24 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:07.562539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 813/1629] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:07 2022] PUT /identity/v3/projects/98b48685bfc94524a3651762260da0c3/users/2b98b3146a4649e28ae48a403062f2d2/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:07.566726 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d6271-a7ff-41c4-a8f2-aa7924382d24 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:07.605044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:07.668187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1830af2-8e70-40bd-a9de-255cad8f7935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:07.669037 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1830af2-8e70-40bd-a9de-255cad8f7935 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:07.669963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1830af2-8e70-40bd-a9de-255cad8f7935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:07.670286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1830af2-8e70-40bd-a9de-255cad8f7935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:07.671036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1830af2-8e70-40bd-a9de-255cad8f7935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:07.672955 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1830af2-8e70-40bd-a9de-255cad8f7935 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:07.692679 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 817/1630] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:07.696672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae743a46-8f03-4739-af11-58f915c35cc8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:07.708074 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae743a46-8f03-4739-af11-58f915c35cc8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:07.715640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22bbec-3aec-41cf-9711-520d384e6523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:07.716489 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22bbec-3aec-41cf-9711-520d384e6523 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:07.719364 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22bbec-3aec-41cf-9711-520d384e6523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:07.719770 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22bbec-3aec-41cf-9711-520d384e6523 admin admin] PATH_INFO: `/v3/projects/98b48685bfc94524a3651762260da0c3/users/2b98b3146a4649e28ae48a403062f2d2/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:07.729790 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22bbec-3aec-41cf-9711-520d384e6523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b48685bfc94524a3651762260da0c3, user_id=2b98b3146a4649e28ae48a403062f2d2, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:07.732974 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22bbec-3aec-41cf-9711-520d384e6523 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:07.756944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 814/1631] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:07 2022] PUT /identity/v3/projects/98b48685bfc94524a3651762260da0c3/users/2b98b3146a4649e28ae48a403062f2d2/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:07.758570 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c33803-99d1-48c3-af5c-e7908e54ded3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:07.758884 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c33803-99d1-48c3-af5c-e7908e54ded3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:07.759188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c33803-99d1-48c3-af5c-e7908e54ded3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:07.803736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b0c33803-99d1-48c3-af5c-e7908e54ded3 None None] MFA Rules not processed for user `2b98b3146a4649e28ae48a403062f2d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:07.805153 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b0c33803-99d1-48c3-af5c-e7908e54ded3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:07.922728 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 818/1632] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:07.935585 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22bbec-3aec-41cf-9711-520d384e6523 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:08.029998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22bbec-3aec-41cf-9711-520d384e6523 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:08.042004 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73c665-951d-4956-9357-bf0711790b75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:08.042743 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73c665-951d-4956-9357-bf0711790b75 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:08.043607 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73c665-951d-4956-9357-bf0711790b75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:08.043896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73c665-951d-4956-9357-bf0711790b75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:08.054564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73c665-951d-4956-9357-bf0711790b75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:08.056098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73c665-951d-4956-9357-bf0711790b75 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:08.061615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:08.088406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 815/1633] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:08.215725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c33803-99d1-48c3-af5c-e7908e54ded3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:08.227992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c33803-99d1-48c3-af5c-e7908e54ded3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:08.232130 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e0699-993f-4e7e-974f-57e0736c6784 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:08.232912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e0699-993f-4e7e-974f-57e0736c6784 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:08.233350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e0699-993f-4e7e-974f-57e0736c6784 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:08.233757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e0699-993f-4e7e-974f-57e0736c6784 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:08.350874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e0699-993f-4e7e-974f-57e0736c6784 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:08.355478 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e0699-993f-4e7e-974f-57e0736c6784 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:08.378456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 819/1634] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:08.436643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73c665-951d-4956-9357-bf0711790b75 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:08.438637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dd73c665-951d-4956-9357-bf0711790b75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:08.592230 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73c665-951d-4956-9357-bf0711790b75 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:08.600290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429a1ff-6893-4168-a979-ecc784393b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:08.601059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429a1ff-6893-4168-a979-ecc784393b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:08.601366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429a1ff-6893-4168-a979-ecc784393b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:08.601679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429a1ff-6893-4168-a979-ecc784393b29 admin admin] PATH_INFO: `/v3/users/4d9d4e9491ff470c903acfffb15caf33` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:08.603277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429a1ff-6893-4168-a979-ecc784393b29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d9d4e9491ff470c903acfffb15caf33)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:08.606899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429a1ff-6893-4168-a979-ecc784393b29 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:08.709258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d429a1ff-6893-4168-a979-ecc784393b29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d9d4e9491ff470c903acfffb15caf33'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:08.732998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d429a1ff-6893-4168-a979-ecc784393b29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d9d4e9491ff470c903acfffb15caf33'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:08.734008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d429a1ff-6893-4168-a979-ecc784393b29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d9d4e9491ff470c903acfffb15caf33'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:08.759209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d429a1ff-6893-4168-a979-ecc784393b29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d9d4e9491ff470c903acfffb15caf33'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:08.770259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d429a1ff-6893-4168-a979-ecc784393b29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d9d4e9491ff470c903acfffb15caf33'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:08.777583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 816/1635] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:08 2022] DELETE /identity/v3/users/4d9d4e9491ff470c903acfffb15caf33 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:08.791311 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b7530-508b-422b-94cd-00f6d208f877 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:08.791311 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b7530-508b-422b-94cd-00f6d208f877 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:08.791311 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b7530-508b-422b-94cd-00f6d208f877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:08.811219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-129b7530-508b-422b-94cd-00f6d208f877 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:08.812739 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-129b7530-508b-422b-94cd-00f6d208f877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:08.870377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:08.894061 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 820/1636] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:08.903586 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429a1ff-6893-4168-a979-ecc784393b29 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:08.905161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d429a1ff-6893-4168-a979-ecc784393b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:08.964885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429a1ff-6893-4168-a979-ecc784393b29 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:08.968399 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd2c8c-047b-4eed-9e97-adef0ee1945a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:08.969014 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd2c8c-047b-4eed-9e97-adef0ee1945a service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:08.969294 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd2c8c-047b-4eed-9e97-adef0ee1945a service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:08.969570 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd2c8c-047b-4eed-9e97-adef0ee1945a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:08.978658 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd2c8c-047b-4eed-9e97-adef0ee1945a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:08.980127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd2c8c-047b-4eed-9e97-adef0ee1945a service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:08.989725 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 817/1637] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:09.266596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b7530-508b-422b-94cd-00f6d208f877 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:09.268078 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-129b7530-508b-422b-94cd-00f6d208f877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:09.284138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b7530-508b-422b-94cd-00f6d208f877 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:09.289338 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:09.290032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:09.290314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:09.290639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] PATH_INFO: `/v3/projects/f7374385eb294799984a042bd17f1ff0` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:09.291890 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7374385eb294799984a042bd17f1ff0)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:09.293740 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:09.349895 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7374385eb294799984a042bd17f1ff0', 'request_id': 'req-26543475-ccea-40c6-9bb7-73e6820d52e1'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:09.356056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7374385eb294799984a042bd17f1ff0', 'request_id': 'req-26543475-ccea-40c6-9bb7-73e6820d52e1'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:09.363571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7374385eb294799984a042bd17f1ff0', 'request_id': 'req-26543475-ccea-40c6-9bb7-73e6820d52e1'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:09.376847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] The token cache is being invalidate because project f7374385eb294799984a042bd17f1ff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:09.377129 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:09.377868 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:09.379506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 821/1638] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:09 2022] DELETE /identity/v3/projects/f7374385eb294799984a042bd17f1ff0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:09.383447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd2c8c-047b-4eed-9e97-adef0ee1945a service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:09.393282 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd2c8c-047b-4eed-9e97-adef0ee1945a service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:09.411975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:09.460892 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee317430-daeb-42c7-a060-bd1fade859c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:09.461595 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee317430-daeb-42c7-a060-bd1fade859c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:09.461910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee317430-daeb-42c7-a060-bd1fade859c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:09.462264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee317430-daeb-42c7-a060-bd1fade859c4 admin admin] PATH_INFO: `/v3/users/dcc27c3de0aa4b6da7325b2ecacd06aa` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:09.464107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee317430-daeb-42c7-a060-bd1fade859c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcc27c3de0aa4b6da7325b2ecacd06aa)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:09.466254 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee317430-daeb-42c7-a060-bd1fade859c4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:09.541004 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ee317430-daeb-42c7-a060-bd1fade859c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcc27c3de0aa4b6da7325b2ecacd06aa'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:09.548530 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ee317430-daeb-42c7-a060-bd1fade859c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcc27c3de0aa4b6da7325b2ecacd06aa'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:09.549153 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ee317430-daeb-42c7-a060-bd1fade859c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcc27c3de0aa4b6da7325b2ecacd06aa'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:09.561326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ee317430-daeb-42c7-a060-bd1fade859c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcc27c3de0aa4b6da7325b2ecacd06aa'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:09.568742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ee317430-daeb-42c7-a060-bd1fade859c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcc27c3de0aa4b6da7325b2ecacd06aa'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:09.574513 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 818/1639] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:09 2022] DELETE /identity/v3/users/dcc27c3de0aa4b6da7325b2ecacd06aa => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:09.853292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:09.855191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:09.888290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26543475-ccea-40c6-9bb7-73e6820d52e1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:09.972366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:09.972366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:09.972366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:09.972366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4 admin admin] PATH_INFO: `/v3/projects/79d967ef48b14dbaacdba8c2cc634e28` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:09.976171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d967ef48b14dbaacdba8c2cc634e28)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:09.978102 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:10.066368 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d967ef48b14dbaacdba8c2cc634e28', 'request_id': 'req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:10.077379 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d967ef48b14dbaacdba8c2cc634e28', 'request_id': 'req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:10.084917 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d967ef48b14dbaacdba8c2cc634e28', 'request_id': 'req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:10.097699 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:10.103972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4 admin admin] The token cache is being invalidate because project 79d967ef48b14dbaacdba8c2cc634e28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:10.105046 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:10.105934 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-71a7f0a3-538f-4d84-bb2f-b486ef1867c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:10.108442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 822/1640] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:09 2022] DELETE /identity/v3/projects/79d967ef48b14dbaacdba8c2cc634e28 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:10.113360 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7365a7-24ad-4a2d-a46b-81882bcfbc3f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:10.113760 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7365a7-24ad-4a2d-a46b-81882bcfbc3f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:10.114171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7365a7-24ad-4a2d-a46b-81882bcfbc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:10.144125 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-6a7365a7-24ad-4a2d-a46b-81882bcfbc3f None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:10.145757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6a7365a7-24ad-4a2d-a46b-81882bcfbc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:10.208093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 819/1641] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:10.216609 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67d8d-2601-4585-bfd1-9078828eca8b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:10.217001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67d8d-2601-4585-bfd1-9078828eca8b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:10.217280 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67d8d-2601-4585-bfd1-9078828eca8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:10.242270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f4b67d8d-2601-4585-bfd1-9078828eca8b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:10.243575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f4b67d8d-2601-4585-bfd1-9078828eca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:10.251672 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 823/1642] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:10.256427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7365a7-24ad-4a2d-a46b-81882bcfbc3f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:10.266904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7365a7-24ad-4a2d-a46b-81882bcfbc3f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:10.271811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d6f9-41f3-4a96-84fe-beb0b68d8027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:10.272406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d6f9-41f3-4a96-84fe-beb0b68d8027 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:10.272677 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d6f9-41f3-4a96-84fe-beb0b68d8027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:10.272963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d6f9-41f3-4a96-84fe-beb0b68d8027 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:10.273596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5d6f9-41f3-4a96-84fe-beb0b68d8027 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:25:10.273919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5d6f9-41f3-4a96-84fe-beb0b68d8027 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:10.275610 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5d6f9-41f3-4a96-84fe-beb0b68d8027 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:10.288876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 820/1643] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:25:10.293483 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b67d8d-2601-4585-bfd1-9078828eca8b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:10.303366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b67d8d-2601-4585-bfd1-9078828eca8b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:10.308363 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b02099-2b50-4b3a-892e-50a82a45fe5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:10.309083 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b02099-2b50-4b3a-892e-50a82a45fe5c admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:10.309369 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b02099-2b50-4b3a-892e-50a82a45fe5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:10.309654 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b02099-2b50-4b3a-892e-50a82a45fe5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:10.310314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b02099-2b50-4b3a-892e-50a82a45fe5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:10.312333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b02099-2b50-4b3a-892e-50a82a45fe5c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:10.343827 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 824/1644] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:10 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:10.352164 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d6f9-41f3-4a96-84fe-beb0b68d8027 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:10.363541 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d6f9-41f3-4a96-84fe-beb0b68d8027 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:10.392725 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:10.430296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee7ed1-40c4-42ed-a46b-230fa81a9558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:10.430916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee7ed1-40c4-42ed-a46b-230fa81a9558 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:10.431246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee7ed1-40c4-42ed-a46b-230fa81a9558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:10.431543 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee7ed1-40c4-42ed-a46b-230fa81a9558 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:10.432199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee7ed1-40c4-42ed-a46b-230fa81a9558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:10.434305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee7ed1-40c4-42ed-a46b-230fa81a9558 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:10.457671 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 821/1645] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:10 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:10.461867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b02099-2b50-4b3a-892e-50a82a45fe5c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:10.472650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b02099-2b50-4b3a-892e-50a82a45fe5c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:10.477184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd4330-f964-4a00-8e9a-2fef60f3e22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:10.477781 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd4330-f964-4a00-8e9a-2fef60f3e22f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:10.478053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd4330-f964-4a00-8e9a-2fef60f3e22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:10.478321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd4330-f964-4a00-8e9a-2fef60f3e22f admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:10.478957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd4330-f964-4a00-8e9a-2fef60f3e22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:10.480574 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd4330-f964-4a00-8e9a-2fef60f3e22f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:10.492290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 825/1646] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:10.496182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee7ed1-40c4-42ed-a46b-230fa81a9558 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:10.507565 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee7ed1-40c4-42ed-a46b-230fa81a9558 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:10.512544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cae1e-d04e-4c79-a9ff-8d6a118a71ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:10.513165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cae1e-d04e-4c79-a9ff-8d6a118a71ed admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:10.513436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cae1e-d04e-4c79-a9ff-8d6a118a71ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:10.513711 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cae1e-d04e-4c79-a9ff-8d6a118a71ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:10.514364 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cae1e-d04e-4c79-a9ff-8d6a118a71ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:10.516080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cae1e-d04e-4c79-a9ff-8d6a118a71ed admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:10.529302 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 822/1647] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:10.532443 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd4330-f964-4a00-8e9a-2fef60f3e22f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:10.542847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd4330-f964-4a00-8e9a-2fef60f3e22f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:10.547502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b55454-d367-458f-8020-85ef8890d6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:10.548242 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b55454-d367-458f-8020-85ef8890d6ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:10.548519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b55454-d367-458f-8020-85ef8890d6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:10.548789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b55454-d367-458f-8020-85ef8890d6ee admin admin] PATH_INFO: `/v3/projects/6c36af4e34b347d19e2486cddf620bcd/users/1f6852259f164ece8867f449e812f7ea/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:10.569870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b55454-d367-458f-8020-85ef8890d6ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c36af4e34b347d19e2486cddf620bcd, user_id=1f6852259f164ece8867f449e812f7ea, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:10.572801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b55454-d367-458f-8020-85ef8890d6ee admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:10.589235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 826/1648] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:10 2022] PUT /identity/v3/projects/6c36af4e34b347d19e2486cddf620bcd/users/1f6852259f164ece8867f449e812f7ea/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:10.600569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622a60b-8059-4eeb-bc57-2c51283b448e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:10.600862 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622a60b-8059-4eeb-bc57-2c51283b448e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:10.601141 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622a60b-8059-4eeb-bc57-2c51283b448e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:10.632180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a622a60b-8059-4eeb-bc57-2c51283b448e None None] MFA Rules not processed for user `1f6852259f164ece8867f449e812f7ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:10.633517 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a622a60b-8059-4eeb-bc57-2c51283b448e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:10.701018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 823/1649] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:10.706279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b55454-d367-458f-8020-85ef8890d6ee admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:10.790356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b55454-d367-458f-8020-85ef8890d6ee admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:10.796025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9913465-fd3f-4319-a08f-9122bc5baff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:10.796662 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9913465-fd3f-4319-a08f-9122bc5baff7 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:10.796942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9913465-fd3f-4319-a08f-9122bc5baff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:10.797215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9913465-fd3f-4319-a08f-9122bc5baff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:10.797862 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9913465-fd3f-4319-a08f-9122bc5baff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:10.799908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9913465-fd3f-4319-a08f-9122bc5baff7 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:10.832177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 827/1650] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:10 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:10.833404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622a60b-8059-4eeb-bc57-2c51283b448e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:10.845033 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622a60b-8059-4eeb-bc57-2c51283b448e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:10.906972 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ce7aa-5ddf-45e5-983b-c28e2207a14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:10.907857 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ce7aa-5ddf-45e5-983b-c28e2207a14f admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:10.908240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ce7aa-5ddf-45e5-983b-c28e2207a14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:10.908586 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ce7aa-5ddf-45e5-983b-c28e2207a14f admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:10.909971 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ce7aa-5ddf-45e5-983b-c28e2207a14f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:10.912105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ce7aa-5ddf-45e5-983b-c28e2207a14f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:10.933855 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 824/1651] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:10 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:10.937890 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9913465-fd3f-4319-a08f-9122bc5baff7 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:10.948775 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9913465-fd3f-4319-a08f-9122bc5baff7 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:10.953511 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd658c-5dad-4d59-b557-0eba60bbe226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:10.954122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd658c-5dad-4d59-b557-0eba60bbe226 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:10.954403 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd658c-5dad-4d59-b557-0eba60bbe226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:10.954678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd658c-5dad-4d59-b557-0eba60bbe226 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:10.955351 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd658c-5dad-4d59-b557-0eba60bbe226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:10.958107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd658c-5dad-4d59-b557-0eba60bbe226 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:10.968694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 828/1652] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:10.972596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ce7aa-5ddf-45e5-983b-c28e2207a14f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:10.982436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ce7aa-5ddf-45e5-983b-c28e2207a14f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:10.986935 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41228b-366c-474d-95ff-5d49f35c4bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:10.987665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41228b-366c-474d-95ff-5d49f35c4bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:10.987946 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41228b-366c-474d-95ff-5d49f35c4bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:10.988219 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41228b-366c-474d-95ff-5d49f35c4bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:10.988853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41228b-366c-474d-95ff-5d49f35c4bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:10.990745 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41228b-366c-474d-95ff-5d49f35c4bd7 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:11.000361 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 825/1653] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:11.005425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd658c-5dad-4d59-b557-0eba60bbe226 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:11.015246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd658c-5dad-4d59-b557-0eba60bbe226 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:11.020435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54a2ca-8019-4edf-a747-692ffec3510f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:11.021084 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54a2ca-8019-4edf-a747-692ffec3510f admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:11.021376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54a2ca-8019-4edf-a747-692ffec3510f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:11.021634 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54a2ca-8019-4edf-a747-692ffec3510f admin admin] PATH_INFO: `/v3/projects/d23ff52ac3ce4ba08c4ce5bc03ea33bd/users/734162f1cf2c4e698715ada3c28e0754/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:11.036166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54a2ca-8019-4edf-a747-692ffec3510f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23ff52ac3ce4ba08c4ce5bc03ea33bd, user_id=734162f1cf2c4e698715ada3c28e0754, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:11.038805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54a2ca-8019-4edf-a747-692ffec3510f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:11.051866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 829/1654] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:11 2022] PUT /identity/v3/projects/d23ff52ac3ce4ba08c4ce5bc03ea33bd/users/734162f1cf2c4e698715ada3c28e0754/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:11.062051 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86186d64-ea72-42d3-ac86-56ddaffb6462 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:11.062369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86186d64-ea72-42d3-ac86-56ddaffb6462 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:11.062647 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86186d64-ea72-42d3-ac86-56ddaffb6462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:11.089874 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:11.093194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-86186d64-ea72-42d3-ac86-56ddaffb6462 None None] MFA Rules not processed for user `734162f1cf2c4e698715ada3c28e0754`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:11.095145 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-86186d64-ea72-42d3-ac86-56ddaffb6462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:11.168853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 826/1655] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:11.169577 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54a2ca-8019-4edf-a747-692ffec3510f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:11.220705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:11.248029 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54a2ca-8019-4edf-a747-692ffec3510f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:11.257417 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64f405-1c6f-458f-8203-183a2e71b4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:11.258312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64f405-1c6f-458f-8203-183a2e71b4fb admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:11.260936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64f405-1c6f-458f-8203-183a2e71b4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:11.261430 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64f405-1c6f-458f-8203-183a2e71b4fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:11.262425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64f405-1c6f-458f-8203-183a2e71b4fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:11.264721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64f405-1c6f-458f-8203-183a2e71b4fb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:11.311269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 830/1656] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:11 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:25:11.317111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86186d64-ea72-42d3-ac86-56ddaffb6462 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:11.343572 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86186d64-ea72-42d3-ac86-56ddaffb6462 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:11.412128 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51568c7-f70a-47c9-a7f9-1b197af7b57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:11.413460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51568c7-f70a-47c9-a7f9-1b197af7b57c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:11.413460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51568c7-f70a-47c9-a7f9-1b197af7b57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:11.413641 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51568c7-f70a-47c9-a7f9-1b197af7b57c admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:11.414426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51568c7-f70a-47c9-a7f9-1b197af7b57c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:11.416440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51568c7-f70a-47c9-a7f9-1b197af7b57c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:11.446258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 827/1657] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:11 2022] POST /identity/v3/users => generated 299 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:11.450654 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64f405-1c6f-458f-8203-183a2e71b4fb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:11.463962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64f405-1c6f-458f-8203-183a2e71b4fb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:11.471201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674300ea-3aa9-46fe-aa1f-5d215629ee54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:11.472451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674300ea-3aa9-46fe-aa1f-5d215629ee54 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:11.472745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674300ea-3aa9-46fe-aa1f-5d215629ee54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:11.473022 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674300ea-3aa9-46fe-aa1f-5d215629ee54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:11.473678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674300ea-3aa9-46fe-aa1f-5d215629ee54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:11.475609 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674300ea-3aa9-46fe-aa1f-5d215629ee54 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:11.494876 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 831/1658] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:11.499537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51568c7-f70a-47c9-a7f9-1b197af7b57c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:11.514896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51568c7-f70a-47c9-a7f9-1b197af7b57c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:11.528189 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c85a0-0516-458a-b28c-bd91b304ecb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:11.529151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c85a0-0516-458a-b28c-bd91b304ecb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:11.529437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c85a0-0516-458a-b28c-bd91b304ecb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:11.529874 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c85a0-0516-458a-b28c-bd91b304ecb5 admin admin] PATH_INFO: `/v3/projects/dd7e0439e9c04515804b9a5cb66f7006/users/a364a6d8d8f742beac8ef355ee99b7de/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:11.534490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75650352-ed5d-4b19-b9e0-ae427d4773ea None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:11.534786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75650352-ed5d-4b19-b9e0-ae427d4773ea None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:11.535146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75650352-ed5d-4b19-b9e0-ae427d4773ea None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:11.536533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 832/1659] 10.222.0.18 () {54 vars in 921 bytes} [Fri Oct 14 02:25:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:25:11.564381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c85a0-0516-458a-b28c-bd91b304ecb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7e0439e9c04515804b9a5cb66f7006, user_id=a364a6d8d8f742beac8ef355ee99b7de, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:11.569209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c85a0-0516-458a-b28c-bd91b304ecb5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:11.598777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 828/1660] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:11 2022] PUT /identity/v3/projects/dd7e0439e9c04515804b9a5cb66f7006/users/a364a6d8d8f742beac8ef355ee99b7de/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:11.599926 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75650352-ed5d-4b19-b9e0-ae427d4773ea None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:11.675188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410f66d-cfac-4823-b126-289a0cb0651c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:11.675769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410f66d-cfac-4823-b126-289a0cb0651c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:11.675984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410f66d-cfac-4823-b126-289a0cb0651c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:11.706382 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a410f66d-cfac-4823-b126-289a0cb0651c None None] MFA Rules not processed for user `de13be55b4034477baf4c029c1a79757`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:11.708207 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a410f66d-cfac-4823-b126-289a0cb0651c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:11.794319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75650352-ed5d-4b19-b9e0-ae427d4773ea None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:11.799271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:11.800121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd773ed-2e15-48e3-b571-12bda989d68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:11.800782 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd773ed-2e15-48e3-b571-12bda989d68c admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:11.801227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd773ed-2e15-48e3-b571-12bda989d68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:11.801547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd773ed-2e15-48e3-b571-12bda989d68c admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:11.802274 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd773ed-2e15-48e3-b571-12bda989d68c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:11.804039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd773ed-2e15-48e3-b571-12bda989d68c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:11.819352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 833/1661] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:25:11.822874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd773ed-2e15-48e3-b571-12bda989d68c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:11.825130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 829/1662] 10.222.0.18 () {58 vars in 1018 bytes} [Fri Oct 14 02:25:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:25:11.839304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410f66d-cfac-4823-b126-289a0cb0651c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:11.841351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a410f66d-cfac-4823-b126-289a0cb0651c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:11.857491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410f66d-cfac-4823-b126-289a0cb0651c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:11.859187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd773ed-2e15-48e3-b571-12bda989d68c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:11.866567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668d9be-c4bb-4c7b-8406-6284e4affd5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:11.876937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668d9be-c4bb-4c7b-8406-6284e4affd5e service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:11.876937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668d9be-c4bb-4c7b-8406-6284e4affd5e service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:11.876937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668d9be-c4bb-4c7b-8406-6284e4affd5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:11.885371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564e088-5a63-44cf-8199-7e4164e53ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:11.886195 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564e088-5a63-44cf-8199-7e4164e53ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:11.886570 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564e088-5a63-44cf-8199-7e4164e53ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:11.887518 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564e088-5a63-44cf-8199-7e4164e53ee6 admin admin] PATH_INFO: `/v3/projects/dd7e0439e9c04515804b9a5cb66f7006/users/a364a6d8d8f742beac8ef355ee99b7de/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:11.889875 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668d9be-c4bb-4c7b-8406-6284e4affd5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:11.891013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668d9be-c4bb-4c7b-8406-6284e4affd5e service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:11.892571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564e088-5a63-44cf-8199-7e4164e53ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7e0439e9c04515804b9a5cb66f7006, user_id=a364a6d8d8f742beac8ef355ee99b7de, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:11.896371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564e088-5a63-44cf-8199-7e4164e53ee6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:11.939119 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 834/1663] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:11 2022] PUT /identity/v3/projects/dd7e0439e9c04515804b9a5cb66f7006/users/a364a6d8d8f742beac8ef355ee99b7de/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:11.939737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 830/1664] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:11.942901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45faf12-34d7-42be-8a12-e3c70ef1167c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:11.943247 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45faf12-34d7-42be-8a12-e3c70ef1167c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:11.943487 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45faf12-34d7-42be-8a12-e3c70ef1167c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:12.018408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b45faf12-34d7-42be-8a12-e3c70ef1167c None None] MFA Rules not processed for user `a364a6d8d8f742beac8ef355ee99b7de`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:12.019900 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b45faf12-34d7-42be-8a12-e3c70ef1167c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:12.164382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 835/1665] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:12.255302 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668d9be-c4bb-4c7b-8406-6284e4affd5e service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:12.256972 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3668d9be-c4bb-4c7b-8406-6284e4affd5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:12.392096 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668d9be-c4bb-4c7b-8406-6284e4affd5e service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:12.461594 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45faf12-34d7-42be-8a12-e3c70ef1167c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:12.463329 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b45faf12-34d7-42be-8a12-e3c70ef1167c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:12.469501 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89bf321-efb3-4b6d-b98b-f5b9fd9bc28b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:12.470754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bf321-efb3-4b6d-b98b-f5b9fd9bc28b service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:12.471223 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bf321-efb3-4b6d-b98b-f5b9fd9bc28b service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:12.472160 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bf321-efb3-4b6d-b98b-f5b9fd9bc28b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:12.521006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:12.554116 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89bf321-efb3-4b6d-b98b-f5b9fd9bc28b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:12.556186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89bf321-efb3-4b6d-b98b-f5b9fd9bc28b service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:12.570568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45faf12-34d7-42be-8a12-e3c70ef1167c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:12.575901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9c9c6-55a2-4f37-94f5-024b1fbc9676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:12.577460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9c9c6-55a2-4f37-94f5-024b1fbc9676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:12.577460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9c9c6-55a2-4f37-94f5-024b1fbc9676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:12.577460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9c9c6-55a2-4f37-94f5-024b1fbc9676 admin admin] PATH_INFO: `/v3/users/53f68132b44649438a3977d02792702b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:12.578700 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9c9c6-55a2-4f37-94f5-024b1fbc9676 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53f68132b44649438a3977d02792702b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:12.581535 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 831/1666] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:12.582834 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9c9c6-55a2-4f37-94f5-024b1fbc9676 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:12.663092 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-64d9c9c6-55a2-4f37-94f5-024b1fbc9676 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53f68132b44649438a3977d02792702b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:12.668879 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-64d9c9c6-55a2-4f37-94f5-024b1fbc9676 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53f68132b44649438a3977d02792702b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:12.682753 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-64d9c9c6-55a2-4f37-94f5-024b1fbc9676 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53f68132b44649438a3977d02792702b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:12.691897 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-64d9c9c6-55a2-4f37-94f5-024b1fbc9676 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53f68132b44649438a3977d02792702b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:12.692651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-64d9c9c6-55a2-4f37-94f5-024b1fbc9676 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53f68132b44649438a3977d02792702b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:12.700130 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 836/1667] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:12 2022] DELETE /identity/v3/users/53f68132b44649438a3977d02792702b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:12.704604 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f476b42-d053-416e-bd10-713e12972458 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:12.704897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f476b42-d053-416e-bd10-713e12972458 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:12.705179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f476b42-d053-416e-bd10-713e12972458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:12.730742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-0f476b42-d053-416e-bd10-713e12972458 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:12.732581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0f476b42-d053-416e-bd10-713e12972458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:12.800340 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:12.812175 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 832/1668] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:12.823403 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9c9c6-55a2-4f37-94f5-024b1fbc9676 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:12.825030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-64d9c9c6-55a2-4f37-94f5-024b1fbc9676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:12.902188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9c9c6-55a2-4f37-94f5-024b1fbc9676 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:12.906919 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e52eed-1dc8-47f6-9c9a-a82ba3fe001c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:12.907765 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e52eed-1dc8-47f6-9c9a-a82ba3fe001c service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:12.908189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e52eed-1dc8-47f6-9c9a-a82ba3fe001c service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:12.908854 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e52eed-1dc8-47f6-9c9a-a82ba3fe001c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:12.937044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e52eed-1dc8-47f6-9c9a-a82ba3fe001c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:12.939188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e52eed-1dc8-47f6-9c9a-a82ba3fe001c service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:12.952563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 837/1669] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:13.228491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f476b42-d053-416e-bd10-713e12972458 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:13.230070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0f476b42-d053-416e-bd10-713e12972458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:13.244364 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f476b42-d053-416e-bd10-713e12972458 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:13.249128 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:13.249902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:13.250227 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:13.250564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] PATH_INFO: `/v3/projects/5e5d76e3b1fe4f59acf33cfaaa1c02f8` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:13.251875 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e5d76e3b1fe4f59acf33cfaaa1c02f8)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:13.253908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:13.299772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e5d76e3b1fe4f59acf33cfaaa1c02f8', 'request_id': 'req-098be1c9-4815-4fe4-be32-1fa9647448ce'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.305157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e5d76e3b1fe4f59acf33cfaaa1c02f8', 'request_id': 'req-098be1c9-4815-4fe4-be32-1fa9647448ce'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.311342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e5d76e3b1fe4f59acf33cfaaa1c02f8', 'request_id': 'req-098be1c9-4815-4fe4-be32-1fa9647448ce'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.326535 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] The token cache is being invalidate because project 5e5d76e3b1fe4f59acf33cfaaa1c02f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:13.326835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.327511 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.329023 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 833/1670] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:13 2022] DELETE /identity/v3/projects/5e5d76e3b1fe4f59acf33cfaaa1c02f8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:13.333622 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e52eed-1dc8-47f6-9c9a-a82ba3fe001c service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:13.343451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e52eed-1dc8-47f6-9c9a-a82ba3fe001c service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:13.404603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:13.419441 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029d82b-4b1f-40e9-a6ee-c18e139de3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:13.420082 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029d82b-4b1f-40e9-a6ee-c18e139de3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:13.420358 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029d82b-4b1f-40e9-a6ee-c18e139de3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:13.420629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029d82b-4b1f-40e9-a6ee-c18e139de3f9 admin admin] PATH_INFO: `/v3/users/2b98b3146a4649e28ae48a403062f2d2` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:13.421795 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4029d82b-4b1f-40e9-a6ee-c18e139de3f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b98b3146a4649e28ae48a403062f2d2)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:13.423836 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4029d82b-4b1f-40e9-a6ee-c18e139de3f9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:13.473823 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4029d82b-4b1f-40e9-a6ee-c18e139de3f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b98b3146a4649e28ae48a403062f2d2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.481210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4029d82b-4b1f-40e9-a6ee-c18e139de3f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b98b3146a4649e28ae48a403062f2d2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.496416 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4029d82b-4b1f-40e9-a6ee-c18e139de3f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b98b3146a4649e28ae48a403062f2d2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.503404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4029d82b-4b1f-40e9-a6ee-c18e139de3f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b98b3146a4649e28ae48a403062f2d2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.504754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4029d82b-4b1f-40e9-a6ee-c18e139de3f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b98b3146a4649e28ae48a403062f2d2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.510639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 838/1671] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:13 2022] DELETE /identity/v3/users/2b98b3146a4649e28ae48a403062f2d2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:13.817294 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:13.818882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:13.833765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098be1c9-4815-4fe4-be32-1fa9647448ce admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:13.895923 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef744dc3-ae48-4086-a0a7-142419bdfb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:13.896565 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef744dc3-ae48-4086-a0a7-142419bdfb3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:13.896842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef744dc3-ae48-4086-a0a7-142419bdfb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:13.897117 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef744dc3-ae48-4086-a0a7-142419bdfb3f admin admin] PATH_INFO: `/v3/projects/98b48685bfc94524a3651762260da0c3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:13.898226 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef744dc3-ae48-4086-a0a7-142419bdfb3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b48685bfc94524a3651762260da0c3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:13.900154 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef744dc3-ae48-4086-a0a7-142419bdfb3f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:13.946331 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ef744dc3-ae48-4086-a0a7-142419bdfb3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b48685bfc94524a3651762260da0c3', 'request_id': 'req-ef744dc3-ae48-4086-a0a7-142419bdfb3f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.952061 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ef744dc3-ae48-4086-a0a7-142419bdfb3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b48685bfc94524a3651762260da0c3', 'request_id': 'req-ef744dc3-ae48-4086-a0a7-142419bdfb3f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.958897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ef744dc3-ae48-4086-a0a7-142419bdfb3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b48685bfc94524a3651762260da0c3', 'request_id': 'req-ef744dc3-ae48-4086-a0a7-142419bdfb3f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.972654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ef744dc3-ae48-4086-a0a7-142419bdfb3f admin admin] The token cache is being invalidate because project 98b48685bfc94524a3651762260da0c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:13.973119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ef744dc3-ae48-4086-a0a7-142419bdfb3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.974257 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ef744dc3-ae48-4086-a0a7-142419bdfb3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:13.975839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 834/1672] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:13 2022] DELETE /identity/v3/projects/98b48685bfc94524a3651762260da0c3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:13.980735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8de44-86fa-43fd-8831-39a257b42e0d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:13.981016 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8de44-86fa-43fd-8831-39a257b42e0d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:13.981290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8de44-86fa-43fd-8831-39a257b42e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.001538 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-23b8de44-86fa-43fd-8831-39a257b42e0d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:14.003584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-23b8de44-86fa-43fd-8831-39a257b42e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:14.062796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 839/1673] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:25:14.071868 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e834d8-370e-41be-8cfb-0e667b21c422 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.072157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e834d8-370e-41be-8cfb-0e667b21c422 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.072436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e834d8-370e-41be-8cfb-0e667b21c422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.092742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f8e834d8-370e-41be-8cfb-0e667b21c422 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:14.093945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f8e834d8-370e-41be-8cfb-0e667b21c422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:14.102272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 835/1674] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:14.106462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8de44-86fa-43fd-8831-39a257b42e0d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:14.116565 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8de44-86fa-43fd-8831-39a257b42e0d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:14.120983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508db833-07a8-444f-957b-a5328985e898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:14.121582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508db833-07a8-444f-957b-a5328985e898 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.121860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508db833-07a8-444f-957b-a5328985e898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.122130 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508db833-07a8-444f-957b-a5328985e898 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.122777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508db833-07a8-444f-957b-a5328985e898 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:25:14.123086 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508db833-07a8-444f-957b-a5328985e898 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:14.124700 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508db833-07a8-444f-957b-a5328985e898 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:14.136538 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 840/1675] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:25:14.140830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e834d8-370e-41be-8cfb-0e667b21c422 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:14.147616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:14.151848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e834d8-370e-41be-8cfb-0e667b21c422 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:14.158974 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917eb05-48c0-4500-8580-648dbfed2907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:14.159822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917eb05-48c0-4500-8580-648dbfed2907 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.160269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917eb05-48c0-4500-8580-648dbfed2907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.160701 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917eb05-48c0-4500-8580-648dbfed2907 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.161540 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917eb05-48c0-4500-8580-648dbfed2907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:14.163763 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917eb05-48c0-4500-8580-648dbfed2907 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:14.190960 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 836/1676] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:14 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:14.195157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508db833-07a8-444f-957b-a5328985e898 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:14.204997 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508db833-07a8-444f-957b-a5328985e898 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:14.256306 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:14.263929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3fa68-afe5-4a8e-b214-6050a677173d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:14.264714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3fa68-afe5-4a8e-b214-6050a677173d admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.265153 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3fa68-afe5-4a8e-b214-6050a677173d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.265464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3fa68-afe5-4a8e-b214-6050a677173d admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.266312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3fa68-afe5-4a8e-b214-6050a677173d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:14.268956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3fa68-afe5-4a8e-b214-6050a677173d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:14.290278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 841/1677] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:14 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:14.294109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917eb05-48c0-4500-8580-648dbfed2907 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:14.304722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917eb05-48c0-4500-8580-648dbfed2907 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:14.309414 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab33811-200a-48f3-8f14-18743ea58400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:14.310025 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab33811-200a-48f3-8f14-18743ea58400 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.310305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab33811-200a-48f3-8f14-18743ea58400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.310607 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab33811-200a-48f3-8f14-18743ea58400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.311348 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab33811-200a-48f3-8f14-18743ea58400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:14.313004 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab33811-200a-48f3-8f14-18743ea58400 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:14.323719 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 837/1678] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:14.327584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3fa68-afe5-4a8e-b214-6050a677173d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:14.337560 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3fa68-afe5-4a8e-b214-6050a677173d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:14.342047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23f658-29c1-4370-be3a-962713651b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:14.342633 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23f658-29c1-4370-be3a-962713651b4a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.342906 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23f658-29c1-4370-be3a-962713651b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.343208 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23f658-29c1-4370-be3a-962713651b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.343840 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23f658-29c1-4370-be3a-962713651b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:14.345409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23f658-29c1-4370-be3a-962713651b4a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:14.355233 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 842/1679] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:25:14.359320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab33811-200a-48f3-8f14-18743ea58400 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:14.370241 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab33811-200a-48f3-8f14-18743ea58400 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:14.374966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b47fb-23b2-45ac-8930-6e1fab7deaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:14.375655 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b47fb-23b2-45ac-8930-6e1fab7deaa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.375960 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b47fb-23b2-45ac-8930-6e1fab7deaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.376263 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b47fb-23b2-45ac-8930-6e1fab7deaa4 admin admin] PATH_INFO: `/v3/projects/5cde3eee7aea42649bbce468003f43bb/users/05167bb4ea654ea89f071c3b7c5fd820/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.392612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b47fb-23b2-45ac-8930-6e1fab7deaa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cde3eee7aea42649bbce468003f43bb, user_id=05167bb4ea654ea89f071c3b7c5fd820, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:14.395342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b47fb-23b2-45ac-8930-6e1fab7deaa4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:14.408472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 838/1680] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:14 2022] PUT /identity/v3/projects/5cde3eee7aea42649bbce468003f43bb/users/05167bb4ea654ea89f071c3b7c5fd820/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:14.419137 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71dba5-8a08-44d5-a751-f8ac538ae223 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.419434 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71dba5-8a08-44d5-a751-f8ac538ae223 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.419715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71dba5-8a08-44d5-a751-f8ac538ae223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.448252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3e71dba5-8a08-44d5-a751-f8ac538ae223 None None] MFA Rules not processed for user `05167bb4ea654ea89f071c3b7c5fd820`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:14.449455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3e71dba5-8a08-44d5-a751-f8ac538ae223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:14.514422 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 843/1681] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:14.519012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b47fb-23b2-45ac-8930-6e1fab7deaa4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:14.579955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b47fb-23b2-45ac-8930-6e1fab7deaa4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:14.585419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de3410-dcbe-49cc-8b02-a080e9fd18ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:14.586032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de3410-dcbe-49cc-8b02-a080e9fd18ad admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.586308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de3410-dcbe-49cc-8b02-a080e9fd18ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.586576 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de3410-dcbe-49cc-8b02-a080e9fd18ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.587333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de3410-dcbe-49cc-8b02-a080e9fd18ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:14.589231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de3410-dcbe-49cc-8b02-a080e9fd18ad admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:14.626448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 839/1682] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:14 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:14.628420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71dba5-8a08-44d5-a751-f8ac538ae223 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:14.639652 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71dba5-8a08-44d5-a751-f8ac538ae223 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:14.700098 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d38012-926d-42d9-ba6d-c1c3983f6cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:14.700748 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d38012-926d-42d9-ba6d-c1c3983f6cff admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.701028 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d38012-926d-42d9-ba6d-c1c3983f6cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.701420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d38012-926d-42d9-ba6d-c1c3983f6cff admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.701947 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d38012-926d-42d9-ba6d-c1c3983f6cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:14.703937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d38012-926d-42d9-ba6d-c1c3983f6cff admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:14.731228 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 844/1683] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:14 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:14.734315 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de3410-dcbe-49cc-8b02-a080e9fd18ad admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:14.745657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de3410-dcbe-49cc-8b02-a080e9fd18ad admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:14.750487 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fec059-4b43-4c56-ae19-dc4f5151b3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:14.751137 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fec059-4b43-4c56-ae19-dc4f5151b3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.751448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fec059-4b43-4c56-ae19-dc4f5151b3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.751724 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fec059-4b43-4c56-ae19-dc4f5151b3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.752913 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fec059-4b43-4c56-ae19-dc4f5151b3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:14.754554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fec059-4b43-4c56-ae19-dc4f5151b3b2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:14.765950 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 840/1684] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:14.770091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d38012-926d-42d9-ba6d-c1c3983f6cff admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:14.780148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d38012-926d-42d9-ba6d-c1c3983f6cff admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:14.785438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5636f13-77e3-4ff1-92ee-95fd899e9979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:14.786100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5636f13-77e3-4ff1-92ee-95fd899e9979 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.786377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5636f13-77e3-4ff1-92ee-95fd899e9979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.786650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5636f13-77e3-4ff1-92ee-95fd899e9979 admin admin] PATH_INFO: `/v3/projects/bd9cc14876104fd09561a159a4b11004/users/93e4f6b14df3455a86cd2b01fd9c06b3/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.802273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5636f13-77e3-4ff1-92ee-95fd899e9979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd9cc14876104fd09561a159a4b11004, user_id=93e4f6b14df3455a86cd2b01fd9c06b3, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:14.805059 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5636f13-77e3-4ff1-92ee-95fd899e9979 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:14.833732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 845/1685] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:14 2022] PUT /identity/v3/projects/bd9cc14876104fd09561a159a4b11004/users/93e4f6b14df3455a86cd2b01fd9c06b3/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:14.837943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fec059-4b43-4c56-ae19-dc4f5151b3b2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:14.848131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fec059-4b43-4c56-ae19-dc4f5151b3b2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:14.910854 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409db8cc-6cca-4c76-aca2-207dff7edd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:14.911781 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409db8cc-6cca-4c76-aca2-207dff7edd2c admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.911781 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409db8cc-6cca-4c76-aca2-207dff7edd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.912050 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409db8cc-6cca-4c76-aca2-207dff7edd2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.912795 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409db8cc-6cca-4c76-aca2-207dff7edd2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:14.920268 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409db8cc-6cca-4c76-aca2-207dff7edd2c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:14.937714 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 841/1686] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:14.939345 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5636f13-77e3-4ff1-92ee-95fd899e9979 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:14.950928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5636f13-77e3-4ff1-92ee-95fd899e9979 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:14.955309 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8755c1b-656d-400c-b993-b93158187e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:14.956584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8755c1b-656d-400c-b993-b93158187e25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.956875 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8755c1b-656d-400c-b993-b93158187e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.957151 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8755c1b-656d-400c-b993-b93158187e25 admin admin] PATH_INFO: `/v3/projects/bd9cc14876104fd09561a159a4b11004/users/93e4f6b14df3455a86cd2b01fd9c06b3/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:14.959230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8755c1b-656d-400c-b993-b93158187e25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd9cc14876104fd09561a159a4b11004, user_id=93e4f6b14df3455a86cd2b01fd9c06b3, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:14.961931 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8755c1b-656d-400c-b993-b93158187e25 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:14.972930 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 846/1687] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:14 2022] PUT /identity/v3/projects/bd9cc14876104fd09561a159a4b11004/users/93e4f6b14df3455a86cd2b01fd9c06b3/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:14.984363 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6c657-0b73-44c1-b2e6-6a9b01580713 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:14.984733 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6c657-0b73-44c1-b2e6-6a9b01580713 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:14.985105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6c657-0b73-44c1-b2e6-6a9b01580713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:15.020760 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4bb6c657-0b73-44c1-b2e6-6a9b01580713 None None] MFA Rules not processed for user `93e4f6b14df3455a86cd2b01fd9c06b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:15.022233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4bb6c657-0b73-44c1-b2e6-6a9b01580713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:15.028734 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:15.118893 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 842/1688] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:15.131040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8755c1b-656d-400c-b993-b93158187e25 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:15.132928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f8755c1b-656d-400c-b993-b93158187e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:15.152950 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:15.196466 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8755c1b-656d-400c-b993-b93158187e25 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:15.200235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c768e1f-be67-4147-b1b7-24970c363c96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:15.200848 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c768e1f-be67-4147-b1b7-24970c363c96 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:15.201152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c768e1f-be67-4147-b1b7-24970c363c96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:15.201425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c768e1f-be67-4147-b1b7-24970c363c96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:15.210273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c768e1f-be67-4147-b1b7-24970c363c96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:15.211739 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c768e1f-be67-4147-b1b7-24970c363c96 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:15.221835 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 847/1689] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:15.584306 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6c657-0b73-44c1-b2e6-6a9b01580713 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:15.645577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6c657-0b73-44c1-b2e6-6a9b01580713 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:15.650092 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9583dba-a1a7-4b50-b966-574101f2d8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:15.650721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9583dba-a1a7-4b50-b966-574101f2d8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:15.651000 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9583dba-a1a7-4b50-b966-574101f2d8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:15.651299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9583dba-a1a7-4b50-b966-574101f2d8e1 admin admin] PATH_INFO: `/v3/users/05167bb4ea654ea89f071c3b7c5fd820` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:15.652427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9583dba-a1a7-4b50-b966-574101f2d8e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05167bb4ea654ea89f071c3b7c5fd820)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:15.654385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9583dba-a1a7-4b50-b966-574101f2d8e1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:15.697529 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c9583dba-a1a7-4b50-b966-574101f2d8e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05167bb4ea654ea89f071c3b7c5fd820'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:15.703859 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c9583dba-a1a7-4b50-b966-574101f2d8e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05167bb4ea654ea89f071c3b7c5fd820'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:15.704544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c9583dba-a1a7-4b50-b966-574101f2d8e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05167bb4ea654ea89f071c3b7c5fd820'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:15.716425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c9583dba-a1a7-4b50-b966-574101f2d8e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05167bb4ea654ea89f071c3b7c5fd820'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:15.724023 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c9583dba-a1a7-4b50-b966-574101f2d8e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05167bb4ea654ea89f071c3b7c5fd820'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:15.729955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 843/1690] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:15 2022] DELETE /identity/v3/users/05167bb4ea654ea89f071c3b7c5fd820 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:15.734626 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeecc07-2674-4fb2-887e-e7df90cdf772 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:15.735050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeecc07-2674-4fb2-887e-e7df90cdf772 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:15.735257 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeecc07-2674-4fb2-887e-e7df90cdf772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:15.755300 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2aeecc07-2674-4fb2-887e-e7df90cdf772 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:15.756505 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2aeecc07-2674-4fb2-887e-e7df90cdf772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:15.813234 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 848/1691] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:15.823627 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9583dba-a1a7-4b50-b966-574101f2d8e1 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:15.825052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c9583dba-a1a7-4b50-b966-574101f2d8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:15.855107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:15.892713 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9583dba-a1a7-4b50-b966-574101f2d8e1 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:15.896420 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff981b-d060-4d65-99dd-01943daf392b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:15.897043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff981b-d060-4d65-99dd-01943daf392b service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:15.897318 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff981b-d060-4d65-99dd-01943daf392b service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:15.897586 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff981b-d060-4d65-99dd-01943daf392b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:15.906973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff981b-d060-4d65-99dd-01943daf392b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:15.908385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff981b-d060-4d65-99dd-01943daf392b service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:15.918523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 844/1692] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:16.372793 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeecc07-2674-4fb2-887e-e7df90cdf772 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:16.373928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2aeecc07-2674-4fb2-887e-e7df90cdf772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:16.388318 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeecc07-2674-4fb2-887e-e7df90cdf772 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:16.394581 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:16.395344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:16.395795 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:16.396245 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] PATH_INFO: `/v3/projects/5cde3eee7aea42649bbce468003f43bb` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:16.397796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cde3eee7aea42649bbce468003f43bb)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:16.400186 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:16.447397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cde3eee7aea42649bbce468003f43bb', 'request_id': 'req-014e341f-6518-4503-9f46-f5342b6f5a6c'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:16.453805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cde3eee7aea42649bbce468003f43bb', 'request_id': 'req-014e341f-6518-4503-9f46-f5342b6f5a6c'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:16.461427 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cde3eee7aea42649bbce468003f43bb', 'request_id': 'req-014e341f-6518-4503-9f46-f5342b6f5a6c'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:16.474582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] The token cache is being invalidate because project 5cde3eee7aea42649bbce468003f43bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:16.474872 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:16.475529 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:16.477042 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 849/1693] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:16 2022] DELETE /identity/v3/projects/5cde3eee7aea42649bbce468003f43bb => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:16.480973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff981b-d060-4d65-99dd-01943daf392b service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:16.493411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff981b-d060-4d65-99dd-01943daf392b service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:16.553599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53ab0c-210b-4201-b2c5-9d1e5804767c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:16.554237 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53ab0c-210b-4201-b2c5-9d1e5804767c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:16.554513 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53ab0c-210b-4201-b2c5-9d1e5804767c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:16.554781 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53ab0c-210b-4201-b2c5-9d1e5804767c admin admin] PATH_INFO: `/v3/users/93e4f6b14df3455a86cd2b01fd9c06b3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:16.555916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53ab0c-210b-4201-b2c5-9d1e5804767c admin admin] RBAC: Authorizing `identity:delete_user(user_id=93e4f6b14df3455a86cd2b01fd9c06b3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:16.557831 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53ab0c-210b-4201-b2c5-9d1e5804767c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:16.604600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd53ab0c-210b-4201-b2c5-9d1e5804767c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93e4f6b14df3455a86cd2b01fd9c06b3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:16.611705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd53ab0c-210b-4201-b2c5-9d1e5804767c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93e4f6b14df3455a86cd2b01fd9c06b3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:16.612515 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd53ab0c-210b-4201-b2c5-9d1e5804767c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93e4f6b14df3455a86cd2b01fd9c06b3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:16.625485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd53ab0c-210b-4201-b2c5-9d1e5804767c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93e4f6b14df3455a86cd2b01fd9c06b3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:16.632522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd53ab0c-210b-4201-b2c5-9d1e5804767c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93e4f6b14df3455a86cd2b01fd9c06b3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:16.638342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 845/1694] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:16 2022] DELETE /identity/v3/users/93e4f6b14df3455a86cd2b01fd9c06b3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:16.920682 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:16.921757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:16.935988 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e341f-6518-4503-9f46-f5342b6f5a6c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:16.981504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:17.000508 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d84ac-7290-4698-90c4-030f427a59b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:17.001179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d84ac-7290-4698-90c4-030f427a59b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:17.001454 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d84ac-7290-4698-90c4-030f427a59b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:17.001726 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d84ac-7290-4698-90c4-030f427a59b9 admin admin] PATH_INFO: `/v3/projects/bd9cc14876104fd09561a159a4b11004` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:17.003679 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d84ac-7290-4698-90c4-030f427a59b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd9cc14876104fd09561a159a4b11004)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:17.005592 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d84ac-7290-4698-90c4-030f427a59b9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:17.053803 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6f2d84ac-7290-4698-90c4-030f427a59b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd9cc14876104fd09561a159a4b11004', 'request_id': 'req-6f2d84ac-7290-4698-90c4-030f427a59b9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:17.059495 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6f2d84ac-7290-4698-90c4-030f427a59b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd9cc14876104fd09561a159a4b11004', 'request_id': 'req-6f2d84ac-7290-4698-90c4-030f427a59b9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:17.065606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6f2d84ac-7290-4698-90c4-030f427a59b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd9cc14876104fd09561a159a4b11004', 'request_id': 'req-6f2d84ac-7290-4698-90c4-030f427a59b9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:17.077804 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6f2d84ac-7290-4698-90c4-030f427a59b9 admin admin] The token cache is being invalidate because project bd9cc14876104fd09561a159a4b11004 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:17.078092 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6f2d84ac-7290-4698-90c4-030f427a59b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:17.078703 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6f2d84ac-7290-4698-90c4-030f427a59b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:17.080219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 850/1695] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:16 2022] DELETE /identity/v3/projects/bd9cc14876104fd09561a159a4b11004 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:17.085355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31435f-635d-4821-8a1a-acd6e2fbcc7c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:17.085831 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31435f-635d-4821-8a1a-acd6e2fbcc7c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:17.085913 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31435f-635d-4821-8a1a-acd6e2fbcc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:17.108348 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4b31435f-635d-4821-8a1a-acd6e2fbcc7c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:17.109531 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4b31435f-635d-4821-8a1a-acd6e2fbcc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:17.164216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:17.187266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 846/1696] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:17.195322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22d14c-9dc6-4e3a-908d-85500fc5eb27 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:17.195610 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22d14c-9dc6-4e3a-908d-85500fc5eb27 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:17.195892 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22d14c-9dc6-4e3a-908d-85500fc5eb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:17.216596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bf22d14c-9dc6-4e3a-908d-85500fc5eb27 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:17.217793 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bf22d14c-9dc6-4e3a-908d-85500fc5eb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:17.227662 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 851/1697] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:17.231684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31435f-635d-4821-8a1a-acd6e2fbcc7c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:17.242798 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31435f-635d-4821-8a1a-acd6e2fbcc7c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:17.250079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807acd6-2839-4dcc-b3e5-3459b2771882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:17.250646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807acd6-2839-4dcc-b3e5-3459b2771882 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:17.251031 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807acd6-2839-4dcc-b3e5-3459b2771882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:17.251446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807acd6-2839-4dcc-b3e5-3459b2771882 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:17.252320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807acd6-2839-4dcc-b3e5-3459b2771882 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:25:17.252701 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807acd6-2839-4dcc-b3e5-3459b2771882 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:17.254491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807acd6-2839-4dcc-b3e5-3459b2771882 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:17.270264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 847/1698] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:25:17.274813 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22d14c-9dc6-4e3a-908d-85500fc5eb27 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:17.285701 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22d14c-9dc6-4e3a-908d-85500fc5eb27 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:17.290585 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b525be-987a-4ae0-b877-40e9a33783cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:17.291723 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b525be-987a-4ae0-b877-40e9a33783cd admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:17.291723 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b525be-987a-4ae0-b877-40e9a33783cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:17.292146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b525be-987a-4ae0-b877-40e9a33783cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:17.293424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b525be-987a-4ae0-b877-40e9a33783cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:17.295154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b525be-987a-4ae0-b877-40e9a33783cd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:17.325431 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 852/1699] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:17 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:25:17.329380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807acd6-2839-4dcc-b3e5-3459b2771882 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:17.343728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807acd6-2839-4dcc-b3e5-3459b2771882 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:17.403900 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8a9da-fdb0-4e35-951a-281e30809781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:17.404569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8a9da-fdb0-4e35-951a-281e30809781 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:17.404870 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8a9da-fdb0-4e35-951a-281e30809781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:17.405167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8a9da-fdb0-4e35-951a-281e30809781 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:17.405822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8a9da-fdb0-4e35-951a-281e30809781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:17.407926 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8a9da-fdb0-4e35-951a-281e30809781 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:17.433419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 848/1700] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:17 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:17.437411 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b525be-987a-4ae0-b877-40e9a33783cd admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:17.449655 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b525be-987a-4ae0-b877-40e9a33783cd admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:17.454599 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce40834-abbe-4496-9bed-7487905f3862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:17.455451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce40834-abbe-4496-9bed-7487905f3862 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:17.455754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce40834-abbe-4496-9bed-7487905f3862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:17.456053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce40834-abbe-4496-9bed-7487905f3862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:17.456749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce40834-abbe-4496-9bed-7487905f3862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:17.458407 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce40834-abbe-4496-9bed-7487905f3862 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:17.469480 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 853/1701] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:17.474075 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8a9da-fdb0-4e35-951a-281e30809781 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:17.485072 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8a9da-fdb0-4e35-951a-281e30809781 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:17.493011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412822c6-db44-448e-b9eb-9081676064d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:17.493863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412822c6-db44-448e-b9eb-9081676064d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:17.494335 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412822c6-db44-448e-b9eb-9081676064d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:17.494784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412822c6-db44-448e-b9eb-9081676064d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:17.495653 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412822c6-db44-448e-b9eb-9081676064d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:17.497458 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412822c6-db44-448e-b9eb-9081676064d7 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:17.509841 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 849/1702] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:17.514141 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce40834-abbe-4496-9bed-7487905f3862 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:17.524907 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce40834-abbe-4496-9bed-7487905f3862 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:17.530133 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df94fc-7694-42ae-a3c8-c1d68cbaee0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:17.530890 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df94fc-7694-42ae-a3c8-c1d68cbaee0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:17.531279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df94fc-7694-42ae-a3c8-c1d68cbaee0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:17.531626 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df94fc-7694-42ae-a3c8-c1d68cbaee0e admin admin] PATH_INFO: `/v3/projects/a67e3aee63a047108d574666eee1028d/users/4352cad0ccfe4debb5fed654ba278953/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:17.547755 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df94fc-7694-42ae-a3c8-c1d68cbaee0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a67e3aee63a047108d574666eee1028d, user_id=4352cad0ccfe4debb5fed654ba278953, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:17.551145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df94fc-7694-42ae-a3c8-c1d68cbaee0e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:17.569502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 854/1703] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:17 2022] PUT /identity/v3/projects/a67e3aee63a047108d574666eee1028d/users/4352cad0ccfe4debb5fed654ba278953/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:17.571018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b0527-f25f-4593-af2a-bfc73cf5487a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:17.571329 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b0527-f25f-4593-af2a-bfc73cf5487a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:17.571602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b0527-f25f-4593-af2a-bfc73cf5487a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:17.592359 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-878b0527-f25f-4593-af2a-bfc73cf5487a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:17.593721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-878b0527-f25f-4593-af2a-bfc73cf5487a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:17.657342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 850/1704] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:25:17.667848 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df94fc-7694-42ae-a3c8-c1d68cbaee0e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:17.669524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-51df94fc-7694-42ae-a3c8-c1d68cbaee0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:17.725575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df94fc-7694-42ae-a3c8-c1d68cbaee0e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:17.729172 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650e1b3-4573-4c97-a0fc-66c5d76710d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:17.729783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e1b3-4573-4c97-a0fc-66c5d76710d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:17.730060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e1b3-4573-4c97-a0fc-66c5d76710d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:17.730323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e1b3-4573-4c97-a0fc-66c5d76710d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:17.741350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650e1b3-4573-4c97-a0fc-66c5d76710d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:17.743032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650e1b3-4573-4c97-a0fc-66c5d76710d5 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:17.753378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 855/1705] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:18.460460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe84fd-3126-4d08-9ff7-fa08c0b33ce7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:18.461469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe84fd-3126-4d08-9ff7-fa08c0b33ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:18.461628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650e1b3-4573-4c97-a0fc-66c5d76710d5 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:18.462031 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe84fd-3126-4d08-9ff7-fa08c0b33ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:18.463281 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5650e1b3-4573-4c97-a0fc-66c5d76710d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:18.491171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:18.528192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7efe84fd-3126-4d08-9ff7-fa08c0b33ce7 None None] MFA Rules not processed for user `4352cad0ccfe4debb5fed654ba278953`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:18.530069 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7efe84fd-3126-4d08-9ff7-fa08c0b33ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:18.564180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650e1b3-4573-4c97-a0fc-66c5d76710d5 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:18.582469 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc972a-ffde-4426-ac61-7a89e8b18627 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:18.583261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc972a-ffde-4426-ac61-7a89e8b18627 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:18.583638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc972a-ffde-4426-ac61-7a89e8b18627 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:18.584003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc972a-ffde-4426-ac61-7a89e8b18627 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:18.587569 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1bcc972a-ffde-4426-ac61-7a89e8b18627 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:18.659909 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:18.781353 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 851/1706] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:18.785560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe84fd-3126-4d08-9ff7-fa08c0b33ce7 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:18.791284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc972a-ffde-4426-ac61-7a89e8b18627 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:18.793646 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc972a-ffde-4426-ac61-7a89e8b18627 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:18.813698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe84fd-3126-4d08-9ff7-fa08c0b33ce7 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:18.819039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 856/1707] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:18.825061 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041697b-071d-4691-98b6-6de761c9ff41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:18.826047 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041697b-071d-4691-98b6-6de761c9ff41 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:18.826256 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041697b-071d-4691-98b6-6de761c9ff41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:18.826532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041697b-071d-4691-98b6-6de761c9ff41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:18.827204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041697b-071d-4691-98b6-6de761c9ff41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:18.829135 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041697b-071d-4691-98b6-6de761c9ff41 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:18.896977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 852/1708] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:18 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:25:18.901184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc972a-ffde-4426-ac61-7a89e8b18627 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:18.923661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc972a-ffde-4426-ac61-7a89e8b18627 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:19.117003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356700ee-0cb6-4889-9363-fb924e8c9c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:19.118244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356700ee-0cb6-4889-9363-fb924e8c9c6f admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:19.119002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356700ee-0cb6-4889-9363-fb924e8c9c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:19.119659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356700ee-0cb6-4889-9363-fb924e8c9c6f admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:19.120807 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356700ee-0cb6-4889-9363-fb924e8c9c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:19.123596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356700ee-0cb6-4889-9363-fb924e8c9c6f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:19.154469 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 857/1709] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:18 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:19.155665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041697b-071d-4691-98b6-6de761c9ff41 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:19.171406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041697b-071d-4691-98b6-6de761c9ff41 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:19.187680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21171ad7-884c-4db8-991b-92f4ab46f688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:19.188483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21171ad7-884c-4db8-991b-92f4ab46f688 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:19.189246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21171ad7-884c-4db8-991b-92f4ab46f688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:19.189246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21171ad7-884c-4db8-991b-92f4ab46f688 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:19.190625 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21171ad7-884c-4db8-991b-92f4ab46f688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:19.192397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21171ad7-884c-4db8-991b-92f4ab46f688 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:19.224578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 853/1710] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:19.227354 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356700ee-0cb6-4889-9363-fb924e8c9c6f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:19.258168 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356700ee-0cb6-4889-9363-fb924e8c9c6f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:19.273300 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652893f6-47ef-4e43-bf27-127dd3ef76e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:19.274938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652893f6-47ef-4e43-bf27-127dd3ef76e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:19.275304 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652893f6-47ef-4e43-bf27-127dd3ef76e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:19.275571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652893f6-47ef-4e43-bf27-127dd3ef76e7 admin admin] PATH_INFO: `/v3/projects/0e8b60a18111456584c1a6e854d39356/users/75dfe1fa72524a13aa6d1367a0391cbb/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:19.321165 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652893f6-47ef-4e43-bf27-127dd3ef76e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8b60a18111456584c1a6e854d39356, user_id=75dfe1fa72524a13aa6d1367a0391cbb, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:19.327924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652893f6-47ef-4e43-bf27-127dd3ef76e7 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:19.364123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 858/1711] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:19 2022] PUT /identity/v3/projects/0e8b60a18111456584c1a6e854d39356/users/75dfe1fa72524a13aa6d1367a0391cbb/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:19.365783 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21171ad7-884c-4db8-991b-92f4ab46f688 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:19.379613 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21171ad7-884c-4db8-991b-92f4ab46f688 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:19.510537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08c4dc-5a12-4de0-a384-58c8fc4be213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:19.512135 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08c4dc-5a12-4de0-a384-58c8fc4be213 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:19.512568 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08c4dc-5a12-4de0-a384-58c8fc4be213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:19.513036 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08c4dc-5a12-4de0-a384-58c8fc4be213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:19.515056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08c4dc-5a12-4de0-a384-58c8fc4be213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:19.545097 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08c4dc-5a12-4de0-a384-58c8fc4be213 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:19.592985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 854/1712] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:19.593434 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652893f6-47ef-4e43-bf27-127dd3ef76e7 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:19.604532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652893f6-47ef-4e43-bf27-127dd3ef76e7 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:19.631023 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b31493-e671-4137-982a-c4f65927f399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:19.631743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b31493-e671-4137-982a-c4f65927f399 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:19.632021 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b31493-e671-4137-982a-c4f65927f399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:19.632292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b31493-e671-4137-982a-c4f65927f399 admin admin] PATH_INFO: `/v3/projects/0e8b60a18111456584c1a6e854d39356/users/75dfe1fa72524a13aa6d1367a0391cbb/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:19.634855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b31493-e671-4137-982a-c4f65927f399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8b60a18111456584c1a6e854d39356, user_id=75dfe1fa72524a13aa6d1367a0391cbb, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:19.637553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b31493-e671-4137-982a-c4f65927f399 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:19.665031 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 859/1713] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:19 2022] PUT /identity/v3/projects/0e8b60a18111456584c1a6e854d39356/users/75dfe1fa72524a13aa6d1367a0391cbb/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:25:19.946283 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08c4dc-5a12-4de0-a384-58c8fc4be213 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:20.121588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08c4dc-5a12-4de0-a384-58c8fc4be213 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:20.201783 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716490c8-2820-4716-8bdd-6c14dfd709a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:20.202290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716490c8-2820-4716-8bdd-6c14dfd709a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:20.202935 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716490c8-2820-4716-8bdd-6c14dfd709a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:20.202935 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716490c8-2820-4716-8bdd-6c14dfd709a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:20.204688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-716490c8-2820-4716-8bdd-6c14dfd709a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:20.222252 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:20.298254 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716490c8-2820-4716-8bdd-6c14dfd709a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:20.300632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716490c8-2820-4716-8bdd-6c14dfd709a5 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:20.314801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 855/1714] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:21.199192 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd5d16-cc2f-46d8-a977-6569c75c5f28 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:21.199192 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd5d16-cc2f-46d8-a977-6569c75c5f28 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:21.199192 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd5d16-cc2f-46d8-a977-6569c75c5f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:21.230110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-22bd5d16-cc2f-46d8-a977-6569c75c5f28 None None] MFA Rules not processed for user `75dfe1fa72524a13aa6d1367a0391cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:21.231404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-22bd5d16-cc2f-46d8-a977-6569c75c5f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:21.239378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:21.319961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 860/1715] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:21.332476 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716490c8-2820-4716-8bdd-6c14dfd709a5 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:21.334438 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-716490c8-2820-4716-8bdd-6c14dfd709a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:21.347595 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716490c8-2820-4716-8bdd-6c14dfd709a5 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:21.352301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58bbd3-c50d-45c0-8e2d-d8387bcbca95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:21.353156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58bbd3-c50d-45c0-8e2d-d8387bcbca95 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:21.353588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58bbd3-c50d-45c0-8e2d-d8387bcbca95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:21.354144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58bbd3-c50d-45c0-8e2d-d8387bcbca95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:21.465021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58bbd3-c50d-45c0-8e2d-d8387bcbca95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:21.470434 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58bbd3-c50d-45c0-8e2d-d8387bcbca95 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:21.504932 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 856/1716] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:27.357714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd5d16-cc2f-46d8-a977-6569c75c5f28 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:27.374549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd5d16-cc2f-46d8-a977-6569c75c5f28 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:27.379243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3af7b7-033f-4f3a-9380-106392d95bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:27.379979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3af7b7-033f-4f3a-9380-106392d95bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:27.380660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3af7b7-033f-4f3a-9380-106392d95bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:27.381296 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3af7b7-033f-4f3a-9380-106392d95bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:27.402036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3af7b7-033f-4f3a-9380-106392d95bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:27.404179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3af7b7-033f-4f3a-9380-106392d95bb1 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:27.429721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 861/1717] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:31.491591 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=51 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:32.237957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58bbd3-c50d-45c0-8e2d-d8387bcbca95 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:32.240661 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3b58bbd3-c50d-45c0-8e2d-d8387bcbca95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:32.355015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58bbd3-c50d-45c0-8e2d-d8387bcbca95 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:32.362730 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d9421-2dc1-4201-b9f3-6a300cf75674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:32.363505 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d9421-2dc1-4201-b9f3-6a300cf75674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:32.363840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d9421-2dc1-4201-b9f3-6a300cf75674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:32.364213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d9421-2dc1-4201-b9f3-6a300cf75674 admin admin] PATH_INFO: `/v3/users/1f6852259f164ece8867f449e812f7ea` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:32.366526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d9421-2dc1-4201-b9f3-6a300cf75674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f6852259f164ece8867f449e812f7ea)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:32.368921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d9421-2dc1-4201-b9f3-6a300cf75674 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:32.449852 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-828d9421-2dc1-4201-b9f3-6a300cf75674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f6852259f164ece8867f449e812f7ea'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:32.460094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-828d9421-2dc1-4201-b9f3-6a300cf75674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f6852259f164ece8867f449e812f7ea'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:32.461277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-828d9421-2dc1-4201-b9f3-6a300cf75674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f6852259f164ece8867f449e812f7ea'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:32.479403 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-828d9421-2dc1-4201-b9f3-6a300cf75674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f6852259f164ece8867f449e812f7ea'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:32.489716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-828d9421-2dc1-4201-b9f3-6a300cf75674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f6852259f164ece8867f449e812f7ea'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:32.498639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 857/1718] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:32 2022] DELETE /identity/v3/users/1f6852259f164ece8867f449e812f7ea => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:32.504753 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f36a5a-5967-48c2-9f9f-b591a72fdb3c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:32.505615 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f36a5a-5967-48c2-9f9f-b591a72fdb3c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:32.506086 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f36a5a-5967-48c2-9f9f-b591a72fdb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:32.533999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b2f36a5a-5967-48c2-9f9f-b591a72fdb3c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:32.544270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b2f36a5a-5967-48c2-9f9f-b591a72fdb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:32.620393 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 862/1719] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:32.630868 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d9421-2dc1-4201-b9f3-6a300cf75674 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:32.632296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-828d9421-2dc1-4201-b9f3-6a300cf75674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:32.690617 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d9421-2dc1-4201-b9f3-6a300cf75674 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:32.694164 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fbc79-cb99-4d9b-a8ef-1b9ecd325d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:32.694770 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fbc79-cb99-4d9b-a8ef-1b9ecd325d55 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:32.695043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fbc79-cb99-4d9b-a8ef-1b9ecd325d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:32.695349 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fbc79-cb99-4d9b-a8ef-1b9ecd325d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:32.704664 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6fbc79-cb99-4d9b-a8ef-1b9ecd325d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:32.706420 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6fbc79-cb99-4d9b-a8ef-1b9ecd325d55 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:32.716407 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 858/1720] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:25:33.071638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f36a5a-5967-48c2-9f9f-b591a72fdb3c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:33.072944 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b2f36a5a-5967-48c2-9f9f-b591a72fdb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:33.085122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f36a5a-5967-48c2-9f9f-b591a72fdb3c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:33.093668 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:33.094391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:33.094786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:33.095254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] PATH_INFO: `/v3/projects/6c36af4e34b347d19e2486cddf620bcd` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:33.096663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c36af4e34b347d19e2486cddf620bcd)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:33.098862 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:33.161734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c36af4e34b347d19e2486cddf620bcd', 'request_id': 'req-7d67a123-8058-412d-8fb0-5876b0125976'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:33.170015 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c36af4e34b347d19e2486cddf620bcd', 'request_id': 'req-7d67a123-8058-412d-8fb0-5876b0125976'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:33.178779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c36af4e34b347d19e2486cddf620bcd', 'request_id': 'req-7d67a123-8058-412d-8fb0-5876b0125976'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:33.199267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] The token cache is being invalidate because project 6c36af4e34b347d19e2486cddf620bcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:33.199267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:33.199267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:33.199267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 863/1721] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:33 2022] DELETE /identity/v3/projects/6c36af4e34b347d19e2486cddf620bcd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:33.201659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fbc79-cb99-4d9b-a8ef-1b9ecd325d55 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:33.215739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fbc79-cb99-4d9b-a8ef-1b9ecd325d55 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:33.251482 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:33.318752 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1183639-affb-4cc3-9c39-0ae181352e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:33.319489 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1183639-affb-4cc3-9c39-0ae181352e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:33.319892 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1183639-affb-4cc3-9c39-0ae181352e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:33.320174 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1183639-affb-4cc3-9c39-0ae181352e61 admin admin] PATH_INFO: `/v3/users/734162f1cf2c4e698715ada3c28e0754` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:33.321318 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1183639-affb-4cc3-9c39-0ae181352e61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=734162f1cf2c4e698715ada3c28e0754)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:33.323301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1183639-affb-4cc3-9c39-0ae181352e61 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:33.376573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e1183639-affb-4cc3-9c39-0ae181352e61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '734162f1cf2c4e698715ada3c28e0754'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:33.386289 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e1183639-affb-4cc3-9c39-0ae181352e61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '734162f1cf2c4e698715ada3c28e0754'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:33.386956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e1183639-affb-4cc3-9c39-0ae181352e61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '734162f1cf2c4e698715ada3c28e0754'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:33.399738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e1183639-affb-4cc3-9c39-0ae181352e61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '734162f1cf2c4e698715ada3c28e0754'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:33.406805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e1183639-affb-4cc3-9c39-0ae181352e61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '734162f1cf2c4e698715ada3c28e0754'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:33.413343 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 859/1722] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:33 2022] DELETE /identity/v3/users/734162f1cf2c4e698715ada3c28e0754 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:33.936168 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:33.937966 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:33.953332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67a123-8058-412d-8fb0-5876b0125976 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:33.964287 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:34.014910 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:34.015821 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:34.016178 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:34.016519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] PATH_INFO: `/v3/projects/d23ff52ac3ce4ba08c4ce5bc03ea33bd` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:34.018343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d23ff52ac3ce4ba08c4ce5bc03ea33bd)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:34.020310 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:34.069930 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd23ff52ac3ce4ba08c4ce5bc03ea33bd', 'request_id': 'req-babbfe70-5cc7-4cee-a1df-2abbea64c669'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.075402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd23ff52ac3ce4ba08c4ce5bc03ea33bd', 'request_id': 'req-babbfe70-5cc7-4cee-a1df-2abbea64c669'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.082180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd23ff52ac3ce4ba08c4ce5bc03ea33bd', 'request_id': 'req-babbfe70-5cc7-4cee-a1df-2abbea64c669'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.094807 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] The token cache is being invalidate because project d23ff52ac3ce4ba08c4ce5bc03ea33bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:34.095093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.095885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.097366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 864/1723] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:33 2022] DELETE /identity/v3/projects/d23ff52ac3ce4ba08c4ce5bc03ea33bd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:34.101527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1183639-affb-4cc3-9c39-0ae181352e61 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:34.102906 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e1183639-affb-4cc3-9c39-0ae181352e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:34.116864 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1183639-affb-4cc3-9c39-0ae181352e61 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:34.203518 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a88bc5-2312-4b86-b04e-23d39f042853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:34.204102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a88bc5-2312-4b86-b04e-23d39f042853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:34.204379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a88bc5-2312-4b86-b04e-23d39f042853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:34.204704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a88bc5-2312-4b86-b04e-23d39f042853 admin admin] PATH_INFO: `/v3/users/a364a6d8d8f742beac8ef355ee99b7de` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:34.205941 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a88bc5-2312-4b86-b04e-23d39f042853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a364a6d8d8f742beac8ef355ee99b7de)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:34.207923 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a88bc5-2312-4b86-b04e-23d39f042853 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:34.253977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:34.257086 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b9a88bc5-2312-4b86-b04e-23d39f042853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a364a6d8d8f742beac8ef355ee99b7de'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.265210 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b9a88bc5-2312-4b86-b04e-23d39f042853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a364a6d8d8f742beac8ef355ee99b7de'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.266191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b9a88bc5-2312-4b86-b04e-23d39f042853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a364a6d8d8f742beac8ef355ee99b7de'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.278904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b9a88bc5-2312-4b86-b04e-23d39f042853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a364a6d8d8f742beac8ef355ee99b7de'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.286243 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b9a88bc5-2312-4b86-b04e-23d39f042853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a364a6d8d8f742beac8ef355ee99b7de'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.292195 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 860/1724] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:34 2022] DELETE /identity/v3/users/a364a6d8d8f742beac8ef355ee99b7de => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:34.625734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:34.627630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:34.650160 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbfe70-5cc7-4cee-a1df-2abbea64c669 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:34.719604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69436d57-ab03-4670-863d-b35264d6be0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:34.720942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69436d57-ab03-4670-863d-b35264d6be0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:34.721226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69436d57-ab03-4670-863d-b35264d6be0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:34.721492 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69436d57-ab03-4670-863d-b35264d6be0c admin admin] PATH_INFO: `/v3/projects/dd7e0439e9c04515804b9a5cb66f7006` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:34.722725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69436d57-ab03-4670-863d-b35264d6be0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd7e0439e9c04515804b9a5cb66f7006)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:34.724865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69436d57-ab03-4670-863d-b35264d6be0c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:34.776614 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:34.780056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69436d57-ab03-4670-863d-b35264d6be0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd7e0439e9c04515804b9a5cb66f7006', 'request_id': 'req-69436d57-ab03-4670-863d-b35264d6be0c'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.788627 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69436d57-ab03-4670-863d-b35264d6be0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd7e0439e9c04515804b9a5cb66f7006', 'request_id': 'req-69436d57-ab03-4670-863d-b35264d6be0c'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.797724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69436d57-ab03-4670-863d-b35264d6be0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd7e0439e9c04515804b9a5cb66f7006', 'request_id': 'req-69436d57-ab03-4670-863d-b35264d6be0c'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.825195 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69436d57-ab03-4670-863d-b35264d6be0c admin admin] The token cache is being invalidate because project dd7e0439e9c04515804b9a5cb66f7006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:34.825497 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69436d57-ab03-4670-863d-b35264d6be0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.826491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69436d57-ab03-4670-863d-b35264d6be0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:34.828278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 865/1725] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:34 2022] DELETE /identity/v3/projects/dd7e0439e9c04515804b9a5cb66f7006 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:34.833328 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f5490-4555-4a31-9643-4747e3ec4e96 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:34.833612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f5490-4555-4a31-9643-4747e3ec4e96 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:34.833891 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f5490-4555-4a31-9643-4747e3ec4e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:34.853592 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-009f5490-4555-4a31-9643-4747e3ec4e96 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:34.854810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-009f5490-4555-4a31-9643-4747e3ec4e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:34.937705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 861/1726] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:25:34.939358 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259a55e-ff3e-49f9-a843-dd625d1b1b6d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:34.939813 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259a55e-ff3e-49f9-a843-dd625d1b1b6d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:34.940058 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259a55e-ff3e-49f9-a843-dd625d1b1b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:34.962824 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4259a55e-ff3e-49f9-a843-dd625d1b1b6d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:34.964221 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4259a55e-ff3e-49f9-a843-dd625d1b1b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:34.979579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 866/1727] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:34.981152 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f5490-4555-4a31-9643-4747e3ec4e96 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:34.995761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f5490-4555-4a31-9643-4747e3ec4e96 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:35.001050 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c578bb4-5c13-44b2-a3ba-b686fddac38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:35.001626 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c578bb4-5c13-44b2-a3ba-b686fddac38d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:35.001983 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c578bb4-5c13-44b2-a3ba-b686fddac38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:35.002346 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c578bb4-5c13-44b2-a3ba-b686fddac38d admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:35.003766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c578bb4-5c13-44b2-a3ba-b686fddac38d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:25:35.003766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c578bb4-5c13-44b2-a3ba-b686fddac38d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:35.006552 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c578bb4-5c13-44b2-a3ba-b686fddac38d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:35.020901 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 862/1728] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:25:35.028252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259a55e-ff3e-49f9-a843-dd625d1b1b6d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:35.041204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259a55e-ff3e-49f9-a843-dd625d1b1b6d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:35.054649 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e68b9e-50d0-48ee-aab8-86e086ec30dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:35.055406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e68b9e-50d0-48ee-aab8-86e086ec30dd admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:35.056079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e68b9e-50d0-48ee-aab8-86e086ec30dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:35.056279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e68b9e-50d0-48ee-aab8-86e086ec30dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:35.057405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e68b9e-50d0-48ee-aab8-86e086ec30dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:35.060428 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e68b9e-50d0-48ee-aab8-86e086ec30dd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:35.122229 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 867/1729] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:35 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:35.128082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c578bb4-5c13-44b2-a3ba-b686fddac38d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:35.143840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c578bb4-5c13-44b2-a3ba-b686fddac38d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:35.169296 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e68b9e-50d0-48ee-aab8-86e086ec30dd admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:35.171661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-72e68b9e-50d0-48ee-aab8-86e086ec30dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:35.279889 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52de7c-36bc-4da7-b355-2f98aa5b8405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:35.280506 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52de7c-36bc-4da7-b355-2f98aa5b8405 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:35.280837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52de7c-36bc-4da7-b355-2f98aa5b8405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:35.281108 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52de7c-36bc-4da7-b355-2f98aa5b8405 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:35.281897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52de7c-36bc-4da7-b355-2f98aa5b8405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:35.283922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52de7c-36bc-4da7-b355-2f98aa5b8405 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:35.331486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e68b9e-50d0-48ee-aab8-86e086ec30dd admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:35.336255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 863/1730] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:35 2022] POST /identity/v3/users => generated 292 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:25:35.337448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb73fb-006f-4993-9fb3-88be94a9c432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:35.338247 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb73fb-006f-4993-9fb3-88be94a9c432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:35.338596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb73fb-006f-4993-9fb3-88be94a9c432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:35.339016 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb73fb-006f-4993-9fb3-88be94a9c432 admin admin] PATH_INFO: `/v3/users/2706c08285894193977e6dd2ed14afc4` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:35.340681 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52de7c-36bc-4da7-b355-2f98aa5b8405 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:35.340804 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb73fb-006f-4993-9fb3-88be94a9c432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2706c08285894193977e6dd2ed14afc4)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:35.343931 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb73fb-006f-4993-9fb3-88be94a9c432 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:35.353996 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52de7c-36bc-4da7-b355-2f98aa5b8405 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:35.359618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfbe9d-9d30-4ca2-9f9a-3bc2df4cb6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:35.360410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfbe9d-9d30-4ca2-9f9a-3bc2df4cb6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:35.361624 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfbe9d-9d30-4ca2-9f9a-3bc2df4cb6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:35.361982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfbe9d-9d30-4ca2-9f9a-3bc2df4cb6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:35.363029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfbe9d-9d30-4ca2-9f9a-3bc2df4cb6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:35.365406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfbe9d-9d30-4ca2-9f9a-3bc2df4cb6a8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:35.383356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 864/1731] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:25:35.387707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfbe9d-9d30-4ca2-9f9a-3bc2df4cb6a8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:35.402559 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfbe9d-9d30-4ca2-9f9a-3bc2df4cb6a8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:35.409435 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f22616-2d1b-47bb-aa23-592a9df5e240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:35.410142 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f22616-2d1b-47bb-aa23-592a9df5e240 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:35.410486 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f22616-2d1b-47bb-aa23-592a9df5e240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:35.411299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f22616-2d1b-47bb-aa23-592a9df5e240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:35.412466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f22616-2d1b-47bb-aa23-592a9df5e240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:35.414524 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f22616-2d1b-47bb-aa23-592a9df5e240 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:35.421175 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6bcb73fb-006f-4993-9fb3-88be94a9c432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2706c08285894193977e6dd2ed14afc4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:35.429360 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 865/1732] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:35.433541 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f22616-2d1b-47bb-aa23-592a9df5e240 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:35.434294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6bcb73fb-006f-4993-9fb3-88be94a9c432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2706c08285894193977e6dd2ed14afc4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:35.444115 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f22616-2d1b-47bb-aa23-592a9df5e240 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:35.448872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf3a73-b6cd-4239-b63d-797463f0f3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:35.449541 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf3a73-b6cd-4239-b63d-797463f0f3d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:35.449914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf3a73-b6cd-4239-b63d-797463f0f3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:35.450090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf3a73-b6cd-4239-b63d-797463f0f3d3 admin admin] PATH_INFO: `/v3/projects/d4b1c4a008a84d9587b98c155ac0ace1/users/ca65b084a0974a5ba648963f71047345/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:35.453750 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6bcb73fb-006f-4993-9fb3-88be94a9c432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2706c08285894193977e6dd2ed14afc4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:35.465009 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6bcb73fb-006f-4993-9fb3-88be94a9c432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2706c08285894193977e6dd2ed14afc4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:35.465847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6bcb73fb-006f-4993-9fb3-88be94a9c432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2706c08285894193977e6dd2ed14afc4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:35.472797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:35.476387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf3a73-b6cd-4239-b63d-797463f0f3d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4b1c4a008a84d9587b98c155ac0ace1, user_id=ca65b084a0974a5ba648963f71047345, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:35.478098 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 868/1733] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:35 2022] DELETE /identity/v3/users/2706c08285894193977e6dd2ed14afc4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:35.479435 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf3a73-b6cd-4239-b63d-797463f0f3d3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:35.480445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fc927-ad0b-4482-9953-acea459f0bb9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:35.480745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fc927-ad0b-4482-9953-acea459f0bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:35.481052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fc927-ad0b-4482-9953-acea459f0bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:35.503720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f35fc927-ad0b-4482-9953-acea459f0bb9 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:35.505126 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f35fc927-ad0b-4482-9953-acea459f0bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:35.509728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 866/1734] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:35 2022] PUT /identity/v3/projects/d4b1c4a008a84d9587b98c155ac0ace1/users/ca65b084a0974a5ba648963f71047345/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:25:35.511174 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be28e0-57a1-4554-bc8b-29506239aae3 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:35.511460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be28e0-57a1-4554-bc8b-29506239aae3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:35.511733 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be28e0-57a1-4554-bc8b-29506239aae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:35.528948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:35.553306 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a6be28e0-57a1-4554-bc8b-29506239aae3 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:35.556656 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a6be28e0-57a1-4554-bc8b-29506239aae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:35.589382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 869/1735] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:35.593780 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fc927-ad0b-4482-9953-acea459f0bb9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:35.595457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f35fc927-ad0b-4482-9953-acea459f0bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:35.607964 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fc927-ad0b-4482-9953-acea459f0bb9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:35.613647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b3d43-03d8-44af-8d1a-927a71ae6bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:35.614361 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b3d43-03d8-44af-8d1a-927a71ae6bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:35.614669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b3d43-03d8-44af-8d1a-927a71ae6bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:35.615383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b3d43-03d8-44af-8d1a-927a71ae6bbd admin admin] PATH_INFO: `/v3/users/3ad09cb4dbd34155bd9ce937c58fc7b5` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:35.616533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b3d43-03d8-44af-8d1a-927a71ae6bbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ad09cb4dbd34155bd9ce937c58fc7b5)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:35.618461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b3d43-03d8-44af-8d1a-927a71ae6bbd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:35.628943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 867/1736] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:35.640148 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be28e0-57a1-4554-bc8b-29506239aae3 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:35.641497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a6be28e0-57a1-4554-bc8b-29506239aae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:35.670652 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-050b3d43-03d8-44af-8d1a-927a71ae6bbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ad09cb4dbd34155bd9ce937c58fc7b5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:35.680949 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-050b3d43-03d8-44af-8d1a-927a71ae6bbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ad09cb4dbd34155bd9ce937c58fc7b5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:35.698087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-050b3d43-03d8-44af-8d1a-927a71ae6bbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ad09cb4dbd34155bd9ce937c58fc7b5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:35.706109 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-050b3d43-03d8-44af-8d1a-927a71ae6bbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ad09cb4dbd34155bd9ce937c58fc7b5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:35.706879 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-050b3d43-03d8-44af-8d1a-927a71ae6bbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ad09cb4dbd34155bd9ce937c58fc7b5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:35.716365 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 870/1737] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:35 2022] DELETE /identity/v3/users/3ad09cb4dbd34155bd9ce937c58fc7b5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:35.717366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b3d43-03d8-44af-8d1a-927a71ae6bbd admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:35.719291 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-050b3d43-03d8-44af-8d1a-927a71ae6bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:35.722369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be28e0-57a1-4554-bc8b-29506239aae3 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:35.802080 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b3d43-03d8-44af-8d1a-927a71ae6bbd admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:35.806696 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361295d-c04e-468a-8da1-0562c08f74d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:35.807488 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361295d-c04e-468a-8da1-0562c08f74d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:35.807766 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361295d-c04e-468a-8da1-0562c08f74d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:35.808054 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361295d-c04e-468a-8da1-0562c08f74d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:35.810038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7361295d-c04e-468a-8da1-0562c08f74d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:35.837708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56568f-4aa0-4c56-9e84-8ecd97d504c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:35.841152 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56568f-4aa0-4c56-9e84-8ecd97d504c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:35.841467 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56568f-4aa0-4c56-9e84-8ecd97d504c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:35.841744 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56568f-4aa0-4c56-9e84-8ecd97d504c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:35.846251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9c56568f-4aa0-4c56-9e84-8ecd97d504c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:35.892530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:35.912608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361295d-c04e-468a-8da1-0562c08f74d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:35.914064 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361295d-c04e-468a-8da1-0562c08f74d1 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:35.925687 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 871/1738] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:35.926973 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ea522-fe70-47b6-8878-52eb2de003b7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:35.927281 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ea522-fe70-47b6-8878-52eb2de003b7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:35.927548 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ea522-fe70-47b6-8878-52eb2de003b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:35.946421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3c1ea522-fe70-47b6-8878-52eb2de003b7 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:35.948006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3c1ea522-fe70-47b6-8878-52eb2de003b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:35.952226 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56568f-4aa0-4c56-9e84-8ecd97d504c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:35.953665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56568f-4aa0-4c56-9e84-8ecd97d504c2 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:35.956624 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 872/1739] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:35.957932 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:35.965665 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ea522-fe70-47b6-8878-52eb2de003b7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:35.966176 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 868/1740] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:25:35.967615 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3c1ea522-fe70-47b6-8878-52eb2de003b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:35.980514 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ea522-fe70-47b6-8878-52eb2de003b7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:35.985435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b54b0c-8136-4a2a-9c60-bc8198596822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:35.986100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b54b0c-8136-4a2a-9c60-bc8198596822 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:35.986377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b54b0c-8136-4a2a-9c60-bc8198596822 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:35.986849 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b54b0c-8136-4a2a-9c60-bc8198596822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:35.997794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b54b0c-8136-4a2a-9c60-bc8198596822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:35.999297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b54b0c-8136-4a2a-9c60-bc8198596822 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:36.010002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 873/1741] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:25:36.651500 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56568f-4aa0-4c56-9e84-8ecd97d504c2 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:36.652993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9c56568f-4aa0-4c56-9e84-8ecd97d504c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:36.666264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56568f-4aa0-4c56-9e84-8ecd97d504c2 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:36.671581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:36.672410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:36.673054 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:36.673054 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2 admin admin] PATH_INFO: `/v3/projects/35b239350d5240bfbe9805986151d082` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:36.674428 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35b239350d5240bfbe9805986151d082)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:36.676511 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:36.761091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35b239350d5240bfbe9805986151d082', 'request_id': 'req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:36.777690 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35b239350d5240bfbe9805986151d082', 'request_id': 'req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:36.788672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35b239350d5240bfbe9805986151d082', 'request_id': 'req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:36.808739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2 admin admin] The token cache is being invalidate because project 35b239350d5240bfbe9805986151d082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:36.809028 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:36.810200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fcbd6df1-13c0-4872-a7e9-8f0f7516f9f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:36.812147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 869/1742] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:36 2022] DELETE /identity/v3/projects/35b239350d5240bfbe9805986151d082 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:36.816704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b54b0c-8136-4a2a-9c60-bc8198596822 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:36.830076 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b54b0c-8136-4a2a-9c60-bc8198596822 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:36.900179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ed1d6-a1e8-41d1-8367-8d653d5b8506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:36.901075 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed1d6-a1e8-41d1-8367-8d653d5b8506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:36.901531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed1d6-a1e8-41d1-8367-8d653d5b8506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:36.901941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed1d6-a1e8-41d1-8367-8d653d5b8506 admin admin] PATH_INFO: `/v3/users/83790aa5a2914dd6ab86f5f3944e7ca3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:36.903319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ed1d6-a1e8-41d1-8367-8d653d5b8506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83790aa5a2914dd6ab86f5f3944e7ca3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:36.905442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ed1d6-a1e8-41d1-8367-8d653d5b8506 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:36.974739 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-956ed1d6-a1e8-41d1-8367-8d653d5b8506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83790aa5a2914dd6ab86f5f3944e7ca3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:36.985145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-956ed1d6-a1e8-41d1-8367-8d653d5b8506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83790aa5a2914dd6ab86f5f3944e7ca3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:37.004193 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-956ed1d6-a1e8-41d1-8367-8d653d5b8506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83790aa5a2914dd6ab86f5f3944e7ca3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:37.013905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-956ed1d6-a1e8-41d1-8367-8d653d5b8506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83790aa5a2914dd6ab86f5f3944e7ca3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:37.014705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-956ed1d6-a1e8-41d1-8367-8d653d5b8506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83790aa5a2914dd6ab86f5f3944e7ca3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:37.023817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 874/1743] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:36 2022] DELETE /identity/v3/users/83790aa5a2914dd6ab86f5f3944e7ca3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:37.045805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1bccc-4f57-473c-b7ac-3d2613158709 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:37.046111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1bccc-4f57-473c-b7ac-3d2613158709 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:37.046378 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1bccc-4f57-473c-b7ac-3d2613158709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:37.083328 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4fa1bccc-4f57-473c-b7ac-3d2613158709 None None] MFA Rules not processed for user `ca65b084a0974a5ba648963f71047345`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:37.084735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4fa1bccc-4f57-473c-b7ac-3d2613158709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:37.189956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 870/1744] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:37.198575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ed1d6-a1e8-41d1-8367-8d653d5b8506 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:37.202073 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-956ed1d6-a1e8-41d1-8367-8d653d5b8506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:37.272817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:37.281151 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ed1d6-a1e8-41d1-8367-8d653d5b8506 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:37.289597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c8865-a48d-4784-899f-cba7f6ae8079 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:37.290330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c8865-a48d-4784-899f-cba7f6ae8079 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:37.290700 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c8865-a48d-4784-899f-cba7f6ae8079 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:37.291045 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c8865-a48d-4784-899f-cba7f6ae8079 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:37.302337 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c8865-a48d-4784-899f-cba7f6ae8079 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:37.305001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c8865-a48d-4784-899f-cba7f6ae8079 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:37.320789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 875/1745] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:37.601390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1bccc-4f57-473c-b7ac-3d2613158709 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:37.603495 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4fa1bccc-4f57-473c-b7ac-3d2613158709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:37.631587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c8865-a48d-4784-899f-cba7f6ae8079 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:37.633030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-eb4c8865-a48d-4784-899f-cba7f6ae8079 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:37.825708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1bccc-4f57-473c-b7ac-3d2613158709 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:37.836318 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c8865-a48d-4784-899f-cba7f6ae8079 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:37.841844 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:37.842658 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:37.843150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:37.843694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36 admin admin] PATH_INFO: `/v3/projects/b7da6346a3734f72914c52dad142e918` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:37.845790 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7da6346a3734f72914c52dad142e918)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:37.848893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:37.864013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:37.864961 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:37.865664 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:37.866170 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21 admin admin] PATH_INFO: `/v3/projects/8d19ceadff46434ea8435f9cb13c2700` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:37.868509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d19ceadff46434ea8435f9cb13c2700)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:37.870820 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:37.921823 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:37.948166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7da6346a3734f72914c52dad142e918', 'request_id': 'req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:37.986512 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7da6346a3734f72914c52dad142e918', 'request_id': 'req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:38.008362 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d19ceadff46434ea8435f9cb13c2700', 'request_id': 'req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:38.010453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7da6346a3734f72914c52dad142e918', 'request_id': 'req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:38.015738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d19ceadff46434ea8435f9cb13c2700', 'request_id': 'req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:38.024249 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d19ceadff46434ea8435f9cb13c2700', 'request_id': 'req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:38.052078 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21 admin admin] The token cache is being invalidate because project 8d19ceadff46434ea8435f9cb13c2700 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:38.052375 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:38.055690 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ab0fc2e1-c600-428c-9397-1ed74b4ceb21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:38.060276 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 871/1746] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:37 2022] DELETE /identity/v3/projects/8d19ceadff46434ea8435f9cb13c2700 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:38.060750 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36 admin admin] The token cache is being invalidate because project b7da6346a3734f72914c52dad142e918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:38.061507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:38.063250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-faf3e3e0-2c47-4cba-bd2e-ac015e71ad36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:38.066076 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b8fbc-f727-40dc-9c57-62122dcd90bf None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.067114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 876/1747] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:37 2022] DELETE /identity/v3/projects/b7da6346a3734f72914c52dad142e918 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:38.067857 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b8fbc-f727-40dc-9c57-62122dcd90bf None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.068232 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b8fbc-f727-40dc-9c57-62122dcd90bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.071736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909f2b3-bafc-49fd-ab79-764e94f525d1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.072026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909f2b3-bafc-49fd-ab79-764e94f525d1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.072506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909f2b3-bafc-49fd-ab79-764e94f525d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.096435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b909f2b3-bafc-49fd-ab79-764e94f525d1 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:38.097815 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b909f2b3-bafc-49fd-ab79-764e94f525d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:38.105517 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e94b8fbc-f727-40dc-9c57-62122dcd90bf None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:38.108096 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e94b8fbc-f727-40dc-9c57-62122dcd90bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:38.176638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 877/1748] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:38.184721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c59228-03bd-4ff5-9e5f-f14988af79a3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.185054 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c59228-03bd-4ff5-9e5f-f14988af79a3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.185288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c59228-03bd-4ff5-9e5f-f14988af79a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.192739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 872/1749] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:25:38.202438 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe55ec-5e92-493d-bfb9-1ab477d2d845 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.202919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe55ec-5e92-493d-bfb9-1ab477d2d845 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.203226 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe55ec-5e92-493d-bfb9-1ab477d2d845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.218412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-70c59228-03bd-4ff5-9e5f-f14988af79a3 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:38.222497 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-70c59228-03bd-4ff5-9e5f-f14988af79a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:38.241218 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b6fe55ec-5e92-493d-bfb9-1ab477d2d845 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:38.243300 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b6fe55ec-5e92-493d-bfb9-1ab477d2d845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:38.247517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 878/1750] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:38.248743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c59228-03bd-4ff5-9e5f-f14988af79a3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:38.260924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 873/1751] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:25:38.267056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe55ec-5e92-493d-bfb9-1ab477d2d845 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:38.268370 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c59228-03bd-4ff5-9e5f-f14988af79a3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:38.281356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe55ec-5e92-493d-bfb9-1ab477d2d845 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:38.286583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c4bc9-711c-4ebe-b979-148581a17a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:38.287285 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c4bc9-711c-4ebe-b979-148581a17a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.287892 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c4bc9-711c-4ebe-b979-148581a17a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.288366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584d53b-dc2b-4c17-8e0b-81e1531227b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:38.289134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584d53b-dc2b-4c17-8e0b-81e1531227b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.289516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584d53b-dc2b-4c17-8e0b-81e1531227b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.289889 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584d53b-dc2b-4c17-8e0b-81e1531227b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.290684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584d53b-dc2b-4c17-8e0b-81e1531227b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:25:38.291145 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584d53b-dc2b-4c17-8e0b-81e1531227b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:38.292044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c4bc9-711c-4ebe-b979-148581a17a6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.293210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c4bc9-711c-4ebe-b979-148581a17a6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:25:38.293481 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584d53b-dc2b-4c17-8e0b-81e1531227b9 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:38.293604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c4bc9-711c-4ebe-b979-148581a17a6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:38.295850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c4bc9-711c-4ebe-b979-148581a17a6b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:38.310179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:38.315027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 879/1752] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:25:38.319416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 874/1753] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:25:38.320631 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584d53b-dc2b-4c17-8e0b-81e1531227b9 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:38.321714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c4bc9-711c-4ebe-b979-148581a17a6b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:38.343894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584d53b-dc2b-4c17-8e0b-81e1531227b9 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:38.345659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c4bc9-711c-4ebe-b979-148581a17a6b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:38.351429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f69cb4-60a3-4e7e-ad1b-846a9bed5502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:38.352075 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f69cb4-60a3-4e7e-ad1b-846a9bed5502 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.352389 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f69cb4-60a3-4e7e-ad1b-846a9bed5502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.352629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f69cb4-60a3-4e7e-ad1b-846a9bed5502 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.354185 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f69cb4-60a3-4e7e-ad1b-846a9bed5502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:38.356591 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f69cb4-60a3-4e7e-ad1b-846a9bed5502 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:38.359638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2279a7-b7b4-4b85-bf9b-1c87181b6772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:38.361417 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2279a7-b7b4-4b85-bf9b-1c87181b6772 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.361794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2279a7-b7b4-4b85-bf9b-1c87181b6772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.362356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2279a7-b7b4-4b85-bf9b-1c87181b6772 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.363301 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2279a7-b7b4-4b85-bf9b-1c87181b6772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:38.365677 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2279a7-b7b4-4b85-bf9b-1c87181b6772 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:38.391155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 875/1754] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:38 2022] POST /identity/v3/projects => generated 391 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:38.395059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f69cb4-60a3-4e7e-ad1b-846a9bed5502 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:38.402288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 880/1755] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:38 2022] POST /identity/v3/projects => generated 383 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:38.408125 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2279a7-b7b4-4b85-bf9b-1c87181b6772 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:38.470986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f69cb4-60a3-4e7e-ad1b-846a9bed5502 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:38.475838 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:38.489481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2279a7-b7b4-4b85-bf9b-1c87181b6772 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:38.499562 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c5da9-ab98-44f9-865d-c9d003b2d0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:38.500688 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c5da9-ab98-44f9-865d-c9d003b2d0cb admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.502211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c5da9-ab98-44f9-865d-c9d003b2d0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.502590 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c5da9-ab98-44f9-865d-c9d003b2d0cb admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.503360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c5da9-ab98-44f9-865d-c9d003b2d0cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:38.505893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c5da9-ab98-44f9-865d-c9d003b2d0cb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:38.541010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 881/1756] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:38 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:38.545586 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c5da9-ab98-44f9-865d-c9d003b2d0cb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:38.547339 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8953404-8093-4256-a553-177f6822261b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:38.547974 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8953404-8093-4256-a553-177f6822261b admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.548795 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8953404-8093-4256-a553-177f6822261b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.549061 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8953404-8093-4256-a553-177f6822261b admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.549748 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8953404-8093-4256-a553-177f6822261b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:38.551914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8953404-8093-4256-a553-177f6822261b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:38.557733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c5da9-ab98-44f9-865d-c9d003b2d0cb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:38.563731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb5c9e-2c80-418e-879e-6a99be1859d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:38.565341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb5c9e-2c80-418e-879e-6a99be1859d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.565900 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb5c9e-2c80-418e-879e-6a99be1859d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.566418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb5c9e-2c80-418e-879e-6a99be1859d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.567236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb5c9e-2c80-418e-879e-6a99be1859d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:38.569190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb5c9e-2c80-418e-879e-6a99be1859d0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:38.579206 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 876/1757] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:38 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:38.583102 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 882/1758] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:38.586247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8953404-8093-4256-a553-177f6822261b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:38.587551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb5c9e-2c80-418e-879e-6a99be1859d0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:38.596355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8953404-8093-4256-a553-177f6822261b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:38.600410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb5c9e-2c80-418e-879e-6a99be1859d0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:38.601223 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6184fe-9439-4096-b554-204d1f8c5b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:38.601847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6184fe-9439-4096-b554-204d1f8c5b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.602175 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6184fe-9439-4096-b554-204d1f8c5b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.602395 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6184fe-9439-4096-b554-204d1f8c5b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.603037 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6184fe-9439-4096-b554-204d1f8c5b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:38.604704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6184fe-9439-4096-b554-204d1f8c5b0b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:38.605990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc5c7a-3d38-4f74-815f-c08efc6085aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:38.606707 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc5c7a-3d38-4f74-815f-c08efc6085aa admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.607121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc5c7a-3d38-4f74-815f-c08efc6085aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.607573 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc5c7a-3d38-4f74-815f-c08efc6085aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.608294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc5c7a-3d38-4f74-815f-c08efc6085aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:38.610014 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc5c7a-3d38-4f74-815f-c08efc6085aa admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:38.616737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 877/1759] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:25:38.631290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6184fe-9439-4096-b554-204d1f8c5b0b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:38.632038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 883/1760] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:25:38.632038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc5c7a-3d38-4f74-815f-c08efc6085aa admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:38.643251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6184fe-9439-4096-b554-204d1f8c5b0b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:38.645443 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc5c7a-3d38-4f74-815f-c08efc6085aa admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:38.650185 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e327501-4ba8-484b-9dfb-16d4e8fcd273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:38.650970 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e327501-4ba8-484b-9dfb-16d4e8fcd273 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.651188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed45bbe-b043-4d7b-8b0e-59982d902ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:38.651472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e327501-4ba8-484b-9dfb-16d4e8fcd273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.651839 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e327501-4ba8-484b-9dfb-16d4e8fcd273 admin admin] PATH_INFO: `/v3/projects/987db81b6d1249d0855576f4d3d6e23d/users/5aa00739773748469374bdd37bdfc3c2/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.651946 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed45bbe-b043-4d7b-8b0e-59982d902ebc admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.652129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed45bbe-b043-4d7b-8b0e-59982d902ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.652411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed45bbe-b043-4d7b-8b0e-59982d902ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.653060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed45bbe-b043-4d7b-8b0e-59982d902ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:38.656182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed45bbe-b043-4d7b-8b0e-59982d902ebc admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:38.669885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 878/1761] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:38.670764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e327501-4ba8-484b-9dfb-16d4e8fcd273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=987db81b6d1249d0855576f4d3d6e23d, user_id=5aa00739773748469374bdd37bdfc3c2, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:38.674031 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e327501-4ba8-484b-9dfb-16d4e8fcd273 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:38.674788 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed45bbe-b043-4d7b-8b0e-59982d902ebc admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:38.686325 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed45bbe-b043-4d7b-8b0e-59982d902ebc admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:38.690293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 884/1762] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:38 2022] PUT /identity/v3/projects/987db81b6d1249d0855576f4d3d6e23d/users/5aa00739773748469374bdd37bdfc3c2/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:38.701408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff01061-f71c-4015-a5f3-9c11c68a182a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.701690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff01061-f71c-4015-a5f3-9c11c68a182a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.701960 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff01061-f71c-4015-a5f3-9c11c68a182a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.733146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9ff01061-f71c-4015-a5f3-9c11c68a182a None None] MFA Rules not processed for user `5aa00739773748469374bdd37bdfc3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:38.734414 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9ff01061-f71c-4015-a5f3-9c11c68a182a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:38.766384 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f813da-2997-48ee-8aa2-f148c83df278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:38.767398 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f813da-2997-48ee-8aa2-f148c83df278 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.767852 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f813da-2997-48ee-8aa2-f148c83df278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.768226 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f813da-2997-48ee-8aa2-f148c83df278 admin admin] PATH_INFO: `/v3/projects/4f63d365cfff4ffe8156ff5ef7f58839/users/024e5ba85c6b438380aea5b469f92137/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.789020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f813da-2997-48ee-8aa2-f148c83df278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f63d365cfff4ffe8156ff5ef7f58839, user_id=024e5ba85c6b438380aea5b469f92137, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:38.792508 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f813da-2997-48ee-8aa2-f148c83df278 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:38.814146 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 879/1763] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:38 2022] PUT /identity/v3/projects/4f63d365cfff4ffe8156ff5ef7f58839/users/024e5ba85c6b438380aea5b469f92137/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:38.826381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b906c0d-89fe-4004-85f8-c9d3874c9449 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.827032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b906c0d-89fe-4004-85f8-c9d3874c9449 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.827032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b906c0d-89fe-4004-85f8-c9d3874c9449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.842964 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 885/1764] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:25:38.844174 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff01061-f71c-4015-a5f3-9c11c68a182a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:38.856206 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff01061-f71c-4015-a5f3-9c11c68a182a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:38.865917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9b906c0d-89fe-4004-85f8-c9d3874c9449 None None] MFA Rules not processed for user `024e5ba85c6b438380aea5b469f92137`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:38.867209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:38.868576 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9b906c0d-89fe-4004-85f8-c9d3874c9449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:38.934111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f897e1-8737-490d-abfe-80895be3b139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:38.935647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f897e1-8737-490d-abfe-80895be3b139 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:38.935933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f897e1-8737-490d-abfe-80895be3b139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:38.936388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f897e1-8737-490d-abfe-80895be3b139 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:38.937050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f897e1-8737-490d-abfe-80895be3b139 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:38.937539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:38.940723 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f897e1-8737-490d-abfe-80895be3b139 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:38.973180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 886/1765] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:38 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:38.978467 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f897e1-8737-490d-abfe-80895be3b139 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:38.992544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 880/1766] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:39.003786 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b906c0d-89fe-4004-85f8-c9d3874c9449 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:39.007326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9b906c0d-89fe-4004-85f8-c9d3874c9449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:39.061995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f897e1-8737-490d-abfe-80895be3b139 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:39.067350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47b294-f8b3-4bfe-9a24-67fbf8798d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:39.068078 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47b294-f8b3-4bfe-9a24-67fbf8798d8f admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:39.068368 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47b294-f8b3-4bfe-9a24-67fbf8798d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:39.068791 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47b294-f8b3-4bfe-9a24-67fbf8798d8f admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:39.069472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47b294-f8b3-4bfe-9a24-67fbf8798d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:39.071770 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47b294-f8b3-4bfe-9a24-67fbf8798d8f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:39.076117 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b906c0d-89fe-4004-85f8-c9d3874c9449 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:39.103319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 887/1767] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:38 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:39.105695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf532068-123b-4f56-9a87-461e7e763ddf None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:39.105695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf532068-123b-4f56-9a87-461e7e763ddf None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:39.105695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf532068-123b-4f56-9a87-461e7e763ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:39.125866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bf532068-123b-4f56-9a87-461e7e763ddf None None] MFA Rules not processed for user `ca65b084a0974a5ba648963f71047345`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:39.127396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bf532068-123b-4f56-9a87-461e7e763ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:39.131186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a36b5c-a8d3-4bb4-af12-b38c9483ad61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:39.132030 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a36b5c-a8d3-4bb4-af12-b38c9483ad61 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:39.132324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a36b5c-a8d3-4bb4-af12-b38c9483ad61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:39.133092 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a36b5c-a8d3-4bb4-af12-b38c9483ad61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:39.194267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 888/1768] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:39.195347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf532068-123b-4f56-9a87-461e7e763ddf None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:39.211645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a36b5c-a8d3-4bb4-af12-b38c9483ad61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:39.213060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a36b5c-a8d3-4bb4-af12-b38c9483ad61 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:39.217998 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf532068-123b-4f56-9a87-461e7e763ddf None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:39.223491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0d339-7215-4d8c-85bb-4aeb5dc87ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:39.223687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 881/1769] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:39 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:39.224047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0d339-7215-4d8c-85bb-4aeb5dc87ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:39.224281 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0d339-7215-4d8c-85bb-4aeb5dc87ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:39.224561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0d339-7215-4d8c-85bb-4aeb5dc87ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:39.225244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0d339-7215-4d8c-85bb-4aeb5dc87ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:39.227000 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0d339-7215-4d8c-85bb-4aeb5dc87ba5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:39.240239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 889/1770] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:39.245190 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a36b5c-a8d3-4bb4-af12-b38c9483ad61 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:39.256870 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a36b5c-a8d3-4bb4-af12-b38c9483ad61 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:39.262164 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62830a7-67d6-463f-bde5-d65311399ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:39.263013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62830a7-67d6-463f-bde5-d65311399ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:39.263564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62830a7-67d6-463f-bde5-d65311399ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:39.263792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62830a7-67d6-463f-bde5-d65311399ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:39.264551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62830a7-67d6-463f-bde5-d65311399ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:39.266257 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62830a7-67d6-463f-bde5-d65311399ff4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:39.279984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 882/1771] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:39.287704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0d339-7215-4d8c-85bb-4aeb5dc87ba5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:39.312700 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0d339-7215-4d8c-85bb-4aeb5dc87ba5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:39.326223 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845cae3-135d-4161-8a75-5dc2e0e04d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:39.326920 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845cae3-135d-4161-8a75-5dc2e0e04d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:39.327765 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845cae3-135d-4161-8a75-5dc2e0e04d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:39.328046 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845cae3-135d-4161-8a75-5dc2e0e04d98 admin admin] PATH_INFO: `/v3/projects/de191e95b74a4a4da6af2318d2d91286/users/3acd7fdb5d8948d0a9761a861a6201c9/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:39.385043 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845cae3-135d-4161-8a75-5dc2e0e04d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de191e95b74a4a4da6af2318d2d91286, user_id=3acd7fdb5d8948d0a9761a861a6201c9, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:39.388025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845cae3-135d-4161-8a75-5dc2e0e04d98 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:39.395805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:39.415115 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 890/1772] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:39 2022] PUT /identity/v3/projects/de191e95b74a4a4da6af2318d2d91286/users/3acd7fdb5d8948d0a9761a861a6201c9/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:39.427506 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651a3bc-fc62-47a7-b79b-1c2e145d1519 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:39.427794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651a3bc-fc62-47a7-b79b-1c2e145d1519 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:39.428071 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651a3bc-fc62-47a7-b79b-1c2e145d1519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:39.508106 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7651a3bc-fc62-47a7-b79b-1c2e145d1519 None None] MFA Rules not processed for user `3acd7fdb5d8948d0a9761a861a6201c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:39.509370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7651a3bc-fc62-47a7-b79b-1c2e145d1519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:39.569554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:39.675132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 883/1773] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:39.679445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845cae3-135d-4161-8a75-5dc2e0e04d98 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:39.877425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845cae3-135d-4161-8a75-5dc2e0e04d98 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:39.883107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a99d8-bfd0-4dba-90e0-81a23bde76de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:39.884176 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a99d8-bfd0-4dba-90e0-81a23bde76de admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:39.884176 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a99d8-bfd0-4dba-90e0-81a23bde76de admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:39.884569 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a99d8-bfd0-4dba-90e0-81a23bde76de admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:39.885445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a99d8-bfd0-4dba-90e0-81a23bde76de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:39.887806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a99d8-bfd0-4dba-90e0-81a23bde76de admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:39.966670 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 891/1774] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:39 2022] POST /identity/v3/projects => generated 385 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:39.971161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651a3bc-fc62-47a7-b79b-1c2e145d1519 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:39.988279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651a3bc-fc62-47a7-b79b-1c2e145d1519 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:40.088020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236523da-1b07-4c93-afe8-03ece50cabf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:40.089533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236523da-1b07-4c93-afe8-03ece50cabf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:40.090013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236523da-1b07-4c93-afe8-03ece50cabf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:40.090363 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236523da-1b07-4c93-afe8-03ece50cabf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:40.091134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236523da-1b07-4c93-afe8-03ece50cabf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:40.093788 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236523da-1b07-4c93-afe8-03ece50cabf9 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:40.130747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 884/1775] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:39 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:40.135237 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a99d8-bfd0-4dba-90e0-81a23bde76de admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:40.150247 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a99d8-bfd0-4dba-90e0-81a23bde76de admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:40.159264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd48d59-ba7b-4875-b958-ccaf9794f855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:40.159929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd48d59-ba7b-4875-b958-ccaf9794f855 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:40.160228 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd48d59-ba7b-4875-b958-ccaf9794f855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:40.160507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd48d59-ba7b-4875-b958-ccaf9794f855 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:40.161181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd48d59-ba7b-4875-b958-ccaf9794f855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:40.163179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd48d59-ba7b-4875-b958-ccaf9794f855 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:40.183391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 892/1776] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:40.187363 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236523da-1b07-4c93-afe8-03ece50cabf9 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:40.198894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236523da-1b07-4c93-afe8-03ece50cabf9 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:40.203936 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c266fc-0c14-4ea6-800f-3844bf74b8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:40.205032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c266fc-0c14-4ea6-800f-3844bf74b8af admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:40.205487 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c266fc-0c14-4ea6-800f-3844bf74b8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:40.205774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c266fc-0c14-4ea6-800f-3844bf74b8af admin admin] PATH_INFO: `/v3/projects/082346d53700453589e502c7c878a679/users/99eddbeda2db4ad29efabcf6672e8f20/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:40.233202 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c266fc-0c14-4ea6-800f-3844bf74b8af admin admin] RBAC: Authorizing `identity:create_grant(project_id=082346d53700453589e502c7c878a679, user_id=99eddbeda2db4ad29efabcf6672e8f20, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:40.237035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c266fc-0c14-4ea6-800f-3844bf74b8af admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:40.274964 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 885/1777] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:40 2022] PUT /identity/v3/projects/082346d53700453589e502c7c878a679/users/99eddbeda2db4ad29efabcf6672e8f20/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:40.279191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd48d59-ba7b-4875-b958-ccaf9794f855 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:40.310867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd48d59-ba7b-4875-b958-ccaf9794f855 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:40.396058 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01fbd6-bf22-4447-8dee-69cf9085ded9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:40.398156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01fbd6-bf22-4447-8dee-69cf9085ded9 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:40.398619 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01fbd6-bf22-4447-8dee-69cf9085ded9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:40.398958 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01fbd6-bf22-4447-8dee-69cf9085ded9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:40.399750 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01fbd6-bf22-4447-8dee-69cf9085ded9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:40.401542 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01fbd6-bf22-4447-8dee-69cf9085ded9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:40.415081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 893/1778] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:40.419480 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c266fc-0c14-4ea6-800f-3844bf74b8af admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:40.431948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c266fc-0c14-4ea6-800f-3844bf74b8af admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:40.437746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40012ca9-21b3-4d02-b268-dd78bb1c3944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:40.438529 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40012ca9-21b3-4d02-b268-dd78bb1c3944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:40.439449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40012ca9-21b3-4d02-b268-dd78bb1c3944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:40.439796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40012ca9-21b3-4d02-b268-dd78bb1c3944 admin admin] PATH_INFO: `/v3/projects/082346d53700453589e502c7c878a679/users/99eddbeda2db4ad29efabcf6672e8f20/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:40.442527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40012ca9-21b3-4d02-b268-dd78bb1c3944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=082346d53700453589e502c7c878a679, user_id=99eddbeda2db4ad29efabcf6672e8f20, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:40.445578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40012ca9-21b3-4d02-b268-dd78bb1c3944 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:40.457683 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 886/1779] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:40 2022] PUT /identity/v3/projects/082346d53700453589e502c7c878a679/users/99eddbeda2db4ad29efabcf6672e8f20/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:25:40.468102 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa7ce9-f6fc-452d-bcd4-ee94063228ff None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:40.468451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa7ce9-f6fc-452d-bcd4-ee94063228ff None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:40.468679 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa7ce9-f6fc-452d-bcd4-ee94063228ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:40.503250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c6aa7ce9-f6fc-452d-bcd4-ee94063228ff None None] MFA Rules not processed for user `99eddbeda2db4ad29efabcf6672e8f20`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:40.504917 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c6aa7ce9-f6fc-452d-bcd4-ee94063228ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:40.579870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:40.607370 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 894/1780] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:40.620186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40012ca9-21b3-4d02-b268-dd78bb1c3944 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:40.621922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-40012ca9-21b3-4d02-b268-dd78bb1c3944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:40.701846 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40012ca9-21b3-4d02-b268-dd78bb1c3944 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:40.709346 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865a3a8-02c8-496b-aa71-a003929420df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:40.710188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865a3a8-02c8-496b-aa71-a003929420df service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:40.710538 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865a3a8-02c8-496b-aa71-a003929420df service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:40.710880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865a3a8-02c8-496b-aa71-a003929420df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:40.721765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865a3a8-02c8-496b-aa71-a003929420df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:40.723260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865a3a8-02c8-496b-aa71-a003929420df service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:40.733823 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 887/1781] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:41.465278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa7ce9-f6fc-452d-bcd4-ee94063228ff None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:41.477067 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa7ce9-f6fc-452d-bcd4-ee94063228ff None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:41.480877 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ae896-7092-49e8-ba08-5f91eb0ad39e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:41.481533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ae896-7092-49e8-ba08-5f91eb0ad39e service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:41.481817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ae896-7092-49e8-ba08-5f91eb0ad39e service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:41.482091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ae896-7092-49e8-ba08-5f91eb0ad39e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:41.557294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ae896-7092-49e8-ba08-5f91eb0ad39e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:41.558743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ae896-7092-49e8-ba08-5f91eb0ad39e service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:41.568937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 895/1782] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:46.848900 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299a890-6b9b-4020-a999-a25118ff2828 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:46.849447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299a890-6b9b-4020-a999-a25118ff2828 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:46.849447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299a890-6b9b-4020-a999-a25118ff2828 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:46.851483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 888/1783] 10.222.0.18 () {54 vars in 921 bytes} [Fri Oct 14 02:25:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:25:46.885019 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740d186-2f35-4b4e-9194-40bce53acdc6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:46.885977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740d186-2f35-4b4e-9194-40bce53acdc6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:46.886437 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740d186-2f35-4b4e-9194-40bce53acdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:46.919818 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0740d186-2f35-4b4e-9194-40bce53acdc6 None None] MFA Rules not processed for user `5e935ba63cdd4256874b99a2eb0a1a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:46.921549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0740d186-2f35-4b4e-9194-40bce53acdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:46.992212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 896/1784] 10.222.0.18 () {58 vars in 1018 bytes} [Fri Oct 14 02:25:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:47.003374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e1645-f7a4-4b9b-ad10-014cacecf8cd None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:47.003760 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e1645-f7a4-4b9b-ad10-014cacecf8cd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:47.004130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e1645-f7a4-4b9b-ad10-014cacecf8cd None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:47.005604 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 889/1785] 10.222.0.18 () {54 vars in 961 bytes} [Fri Oct 14 02:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:25:47.012376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6df96-546f-4ae5-aac4-722ca97615bb None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:47.012663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6df96-546f-4ae5-aac4-722ca97615bb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:47.012940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6df96-546f-4ae5-aac4-722ca97615bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:47.035623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-aee6df96-546f-4ae5-aac4-722ca97615bb None None] MFA Rules not processed for user `5e935ba63cdd4256874b99a2eb0a1a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:47.037034 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-aee6df96-546f-4ae5-aac4-722ca97615bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:47.047959 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 897/1786] 10.222.0.18 () {58 vars in 1058 bytes} [Fri Oct 14 02:25:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:47.057640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e1645-f7a4-4b9b-ad10-014cacecf8cd None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:47.073200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e1645-f7a4-4b9b-ad10-014cacecf8cd None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:47.078421 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe22bd7-08af-4a41-babe-61af49f71b1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e935ba63cdd4256874b99a2eb0a1a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:47.079238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe22bd7-08af-4a41-babe-61af49f71b1a service placement] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:47.079608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe22bd7-08af-4a41-babe-61af49f71b1a service placement] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:47.079991 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe22bd7-08af-4a41-babe-61af49f71b1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:47.088886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe22bd7-08af-4a41-babe-61af49f71b1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:47.090605 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe22bd7-08af-4a41-babe-61af49f71b1a service placement] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:47.101286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 890/1787] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:47.630754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef4331-5e16-4933-a6c0-966744a8d858 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:47.631037 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef4331-5e16-4933-a6c0-966744a8d858 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:47.631332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef4331-5e16-4933-a6c0-966744a8d858 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:47.632696 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 898/1788] 10.222.0.18 () {54 vars in 921 bytes} [Fri Oct 14 02:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:25:47.647827 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13da20-bf8e-4eae-9d29-6fe29d701384 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:47.648226 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13da20-bf8e-4eae-9d29-6fe29d701384 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:47.648606 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13da20-bf8e-4eae-9d29-6fe29d701384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:47.679773 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-eb13da20-bf8e-4eae-9d29-6fe29d701384 None None] MFA Rules not processed for user `5e935ba63cdd4256874b99a2eb0a1a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:47.681046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-eb13da20-bf8e-4eae-9d29-6fe29d701384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:47.692606 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 891/1789] 10.222.0.18 () {58 vars in 1018 bytes} [Fri Oct 14 02:25:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:47.705226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef4331-5e16-4933-a6c0-966744a8d858 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:47.712001 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13da20-bf8e-4eae-9d29-6fe29d701384 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:47.714035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-eb13da20-bf8e-4eae-9d29-6fe29d701384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:47.719594 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef4331-5e16-4933-a6c0-966744a8d858 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:47.727532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3173a0-1381-4328-8531-0b1a03ff026c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e935ba63cdd4256874b99a2eb0a1a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:47.728233 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3173a0-1381-4328-8531-0b1a03ff026c service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:47.728422 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3173a0-1381-4328-8531-0b1a03ff026c service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:47.728694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3173a0-1381-4328-8531-0b1a03ff026c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:47.731826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:47.737651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3173a0-1381-4328-8531-0b1a03ff026c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:47.739277 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3173a0-1381-4328-8531-0b1a03ff026c service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:47.750153 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 899/1790] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:47.799217 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13da20-bf8e-4eae-9d29-6fe29d701384 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:47.803960 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df6fda-dc2b-4a7d-a17b-867e17653f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:47.804605 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df6fda-dc2b-4a7d-a17b-867e17653f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:47.804883 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df6fda-dc2b-4a7d-a17b-867e17653f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:47.805157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df6fda-dc2b-4a7d-a17b-867e17653f3d admin admin] PATH_INFO: `/v3/users/4352cad0ccfe4debb5fed654ba278953` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:47.806294 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df6fda-dc2b-4a7d-a17b-867e17653f3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4352cad0ccfe4debb5fed654ba278953)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:47.808261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df6fda-dc2b-4a7d-a17b-867e17653f3d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:47.862439 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-77df6fda-dc2b-4a7d-a17b-867e17653f3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4352cad0ccfe4debb5fed654ba278953'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:47.870537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-77df6fda-dc2b-4a7d-a17b-867e17653f3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4352cad0ccfe4debb5fed654ba278953'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:47.871224 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-77df6fda-dc2b-4a7d-a17b-867e17653f3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4352cad0ccfe4debb5fed654ba278953'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:47.884001 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-77df6fda-dc2b-4a7d-a17b-867e17653f3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4352cad0ccfe4debb5fed654ba278953'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:47.890545 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-77df6fda-dc2b-4a7d-a17b-867e17653f3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4352cad0ccfe4debb5fed654ba278953'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:47.897387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 892/1791] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:47 2022] DELETE /identity/v3/users/4352cad0ccfe4debb5fed654ba278953 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:25:48.064903 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3173a0-1381-4328-8531-0b1a03ff026c service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:48.080751 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3173a0-1381-4328-8531-0b1a03ff026c service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:48.137671 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:48.138299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:48.138573 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:48.138843 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] PATH_INFO: `/v3/projects/a67e3aee63a047108d574666eee1028d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:48.139969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a67e3aee63a047108d574666eee1028d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:48.142205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:48.150064 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:48.194259 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a67e3aee63a047108d574666eee1028d', 'request_id': 'req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.200172 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a67e3aee63a047108d574666eee1028d', 'request_id': 'req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.206699 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a67e3aee63a047108d574666eee1028d', 'request_id': 'req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.221781 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] The token cache is being invalidate because project a67e3aee63a047108d574666eee1028d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:48.222070 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.222700 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.224288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 900/1792] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:48 2022] DELETE /identity/v3/projects/a67e3aee63a047108d574666eee1028d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:48.228265 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df6fda-dc2b-4a7d-a17b-867e17653f3d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:48.229834 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-77df6fda-dc2b-4a7d-a17b-867e17653f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:48.251708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df6fda-dc2b-4a7d-a17b-867e17653f3d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:48.313021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b85659-cad5-4045-8095-5dab60b0f4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:48.314320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b85659-cad5-4045-8095-5dab60b0f4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:48.314649 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b85659-cad5-4045-8095-5dab60b0f4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:48.314930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b85659-cad5-4045-8095-5dab60b0f4ef admin admin] PATH_INFO: `/v3/users/75dfe1fa72524a13aa6d1367a0391cbb` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:48.316120 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b85659-cad5-4045-8095-5dab60b0f4ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=75dfe1fa72524a13aa6d1367a0391cbb)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:48.318059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b85659-cad5-4045-8095-5dab60b0f4ef admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:48.353134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:48.367896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c5b85659-cad5-4045-8095-5dab60b0f4ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75dfe1fa72524a13aa6d1367a0391cbb'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.374503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c5b85659-cad5-4045-8095-5dab60b0f4ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75dfe1fa72524a13aa6d1367a0391cbb'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.375127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c5b85659-cad5-4045-8095-5dab60b0f4ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75dfe1fa72524a13aa6d1367a0391cbb'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.387466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c5b85659-cad5-4045-8095-5dab60b0f4ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75dfe1fa72524a13aa6d1367a0391cbb'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.394556 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c5b85659-cad5-4045-8095-5dab60b0f4ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75dfe1fa72524a13aa6d1367a0391cbb'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.400534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 893/1793] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:48 2022] DELETE /identity/v3/users/75dfe1fa72524a13aa6d1367a0391cbb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:48.570620 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:48.572082 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:48.585006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fae3f-5c57-4ca2-aa31-b6f23f1407c4 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:48.640972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a90d5-e861-40e1-8172-a6965ccb9462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:48.642249 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a90d5-e861-40e1-8172-a6965ccb9462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:48.642524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a90d5-e861-40e1-8172-a6965ccb9462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:48.642790 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a90d5-e861-40e1-8172-a6965ccb9462 admin admin] PATH_INFO: `/v3/projects/0e8b60a18111456584c1a6e854d39356` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:48.643953 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a90d5-e861-40e1-8172-a6965ccb9462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e8b60a18111456584c1a6e854d39356)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:48.645900 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a90d5-e861-40e1-8172-a6965ccb9462 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:48.692370 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-412a90d5-e861-40e1-8172-a6965ccb9462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e8b60a18111456584c1a6e854d39356', 'request_id': 'req-412a90d5-e861-40e1-8172-a6965ccb9462'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.697818 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-412a90d5-e861-40e1-8172-a6965ccb9462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e8b60a18111456584c1a6e854d39356', 'request_id': 'req-412a90d5-e861-40e1-8172-a6965ccb9462'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.705010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-412a90d5-e861-40e1-8172-a6965ccb9462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e8b60a18111456584c1a6e854d39356', 'request_id': 'req-412a90d5-e861-40e1-8172-a6965ccb9462'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.717408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-412a90d5-e861-40e1-8172-a6965ccb9462 admin admin] The token cache is being invalidate because project 0e8b60a18111456584c1a6e854d39356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:48.717699 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-412a90d5-e861-40e1-8172-a6965ccb9462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.718336 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-412a90d5-e861-40e1-8172-a6965ccb9462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:48.719926 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 901/1794] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:48 2022] DELETE /identity/v3/projects/0e8b60a18111456584c1a6e854d39356 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:48.724922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2483c6-f1fa-4a1e-8c8a-6e24bc5a57f5 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:48.725395 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2483c6-f1fa-4a1e-8c8a-6e24bc5a57f5 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:48.725849 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2483c6-f1fa-4a1e-8c8a-6e24bc5a57f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:48.748992 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-0e2483c6-f1fa-4a1e-8c8a-6e24bc5a57f5 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:48.750299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0e2483c6-f1fa-4a1e-8c8a-6e24bc5a57f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:48.820269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 894/1795] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:48.828040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc79a0d-525c-4da4-999d-b8e3a1d3cc15 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:48.828319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc79a0d-525c-4da4-999d-b8e3a1d3cc15 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:48.828590 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc79a0d-525c-4da4-999d-b8e3a1d3cc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:48.852376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bbc79a0d-525c-4da4-999d-b8e3a1d3cc15 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:48.853600 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bbc79a0d-525c-4da4-999d-b8e3a1d3cc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:48.861998 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 902/1796] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:25:48.866424 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2483c6-f1fa-4a1e-8c8a-6e24bc5a57f5 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:48.877257 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2483c6-f1fa-4a1e-8c8a-6e24bc5a57f5 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:48.881827 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab57f6-9df8-4059-b2ac-3348533f4155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:48.882437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab57f6-9df8-4059-b2ac-3348533f4155 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:48.882713 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab57f6-9df8-4059-b2ac-3348533f4155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:48.882982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab57f6-9df8-4059-b2ac-3348533f4155 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:48.883672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab57f6-9df8-4059-b2ac-3348533f4155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:25:48.883961 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab57f6-9df8-4059-b2ac-3348533f4155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:48.885554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab57f6-9df8-4059-b2ac-3348533f4155 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:48.898169 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 895/1797] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:25:48.902457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc79a0d-525c-4da4-999d-b8e3a1d3cc15 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:48.912586 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc79a0d-525c-4da4-999d-b8e3a1d3cc15 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:48.917446 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29ebb4-a095-4e2a-a23b-36d679632f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:48.918053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29ebb4-a095-4e2a-a23b-36d679632f7a admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:48.918326 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29ebb4-a095-4e2a-a23b-36d679632f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:48.918717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29ebb4-a095-4e2a-a23b-36d679632f7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:48.919379 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29ebb4-a095-4e2a-a23b-36d679632f7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:48.921261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29ebb4-a095-4e2a-a23b-36d679632f7a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:48.943103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:48.951871 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 903/1798] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:25:48 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:48.956556 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab57f6-9df8-4059-b2ac-3348533f4155 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:48.967015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab57f6-9df8-4059-b2ac-3348533f4155 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:49.023208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c806e-6a8b-49a8-bcae-7b3fc58ffa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:49.023823 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c806e-6a8b-49a8-bcae-7b3fc58ffa46 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:49.024096 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c806e-6a8b-49a8-bcae-7b3fc58ffa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:49.024367 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c806e-6a8b-49a8-bcae-7b3fc58ffa46 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:49.024982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c806e-6a8b-49a8-bcae-7b3fc58ffa46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:49.026885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c806e-6a8b-49a8-bcae-7b3fc58ffa46 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:49.050704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 896/1799] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:25:48 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:25:49.054590 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29ebb4-a095-4e2a-a23b-36d679632f7a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:49.064856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29ebb4-a095-4e2a-a23b-36d679632f7a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:49.069718 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec4f4f-2da4-4134-b347-c19e7ea4b1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:49.070471 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec4f4f-2da4-4134-b347-c19e7ea4b1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:49.070754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec4f4f-2da4-4134-b347-c19e7ea4b1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:49.071336 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec4f4f-2da4-4134-b347-c19e7ea4b1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:49.072156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec4f4f-2da4-4134-b347-c19e7ea4b1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:49.074181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec4f4f-2da4-4134-b347-c19e7ea4b1d5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:49.085167 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 904/1800] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:49.089258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c806e-6a8b-49a8-bcae-7b3fc58ffa46 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:49.100154 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c806e-6a8b-49a8-bcae-7b3fc58ffa46 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:49.105042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838db66-4c45-48c4-b5a7-bc0ff01a95e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:49.105765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838db66-4c45-48c4-b5a7-bc0ff01a95e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:49.106046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838db66-4c45-48c4-b5a7-bc0ff01a95e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:49.106316 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838db66-4c45-48c4-b5a7-bc0ff01a95e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:49.106952 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838db66-4c45-48c4-b5a7-bc0ff01a95e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:49.108586 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838db66-4c45-48c4-b5a7-bc0ff01a95e1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:49.119757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 897/1801] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:25:49.123826 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec4f4f-2da4-4134-b347-c19e7ea4b1d5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:49.134431 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec4f4f-2da4-4134-b347-c19e7ea4b1d5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:49.138922 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e3be4-8439-4fab-a6d4-52ba5dc1ec34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:49.139595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e3be4-8439-4fab-a6d4-52ba5dc1ec34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:49.139871 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e3be4-8439-4fab-a6d4-52ba5dc1ec34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:49.140140 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e3be4-8439-4fab-a6d4-52ba5dc1ec34 admin admin] PATH_INFO: `/v3/projects/ca3c42b1d43f49e3940ff089c2de1bd8/users/b1d3ef1211ee48b3946ae418fb425038/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:49.155023 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e3be4-8439-4fab-a6d4-52ba5dc1ec34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca3c42b1d43f49e3940ff089c2de1bd8, user_id=b1d3ef1211ee48b3946ae418fb425038, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:49.157912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e3be4-8439-4fab-a6d4-52ba5dc1ec34 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:49.170436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 905/1802] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:25:49 2022] PUT /identity/v3/projects/ca3c42b1d43f49e3940ff089c2de1bd8/users/b1d3ef1211ee48b3946ae418fb425038/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:49.182295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271ba78-215d-4096-8932-4920ba8368ab None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:49.182585 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271ba78-215d-4096-8932-4920ba8368ab None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:49.182861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271ba78-215d-4096-8932-4920ba8368ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:49.211182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d271ba78-215d-4096-8932-4920ba8368ab None None] MFA Rules not processed for user `b1d3ef1211ee48b3946ae418fb425038`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:49.212384 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d271ba78-215d-4096-8932-4920ba8368ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:49.216264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:49.280337 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 898/1803] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:49.292097 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e3be4-8439-4fab-a6d4-52ba5dc1ec34 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:49.294494 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3c5e3be4-8439-4fab-a6d4-52ba5dc1ec34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:49.356690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e3be4-8439-4fab-a6d4-52ba5dc1ec34 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:49.363910 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa188c-577c-4681-a633-a2654d27f40c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:49.364611 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa188c-577c-4681-a633-a2654d27f40c service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:49.365250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa188c-577c-4681-a633-a2654d27f40c service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:49.365593 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa188c-577c-4681-a633-a2654d27f40c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:49.375334 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa188c-577c-4681-a633-a2654d27f40c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:49.376978 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa188c-577c-4681-a633-a2654d27f40c service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:49.386134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 906/1804] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:50.184453 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271ba78-215d-4096-8932-4920ba8368ab None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:50.194028 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271ba78-215d-4096-8932-4920ba8368ab None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:50.197695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9e92b-0650-4868-ba1f-e2b842ed78c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:50.198307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9e92b-0650-4868-ba1f-e2b842ed78c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:50.198588 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9e92b-0650-4868-ba1f-e2b842ed78c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:50.198859 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9e92b-0650-4868-ba1f-e2b842ed78c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:50.200639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bdd9e92b-0650-4868-ba1f-e2b842ed78c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:50.261559 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9e92b-0650-4868-ba1f-e2b842ed78c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:50.262993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9e92b-0650-4868-ba1f-e2b842ed78c7 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:50.273800 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 899/1805] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:58.397576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa188c-577c-4681-a633-a2654d27f40c service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:58.398977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3ffa188c-577c-4681-a633-a2654d27f40c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:58.469906 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa188c-577c-4681-a633-a2654d27f40c service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:58.474377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014ac4c-c101-49ff-9cb8-43af2254f0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:58.475006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014ac4c-c101-49ff-9cb8-43af2254f0ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:58.475314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014ac4c-c101-49ff-9cb8-43af2254f0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:58.475584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014ac4c-c101-49ff-9cb8-43af2254f0ea admin admin] PATH_INFO: `/v3/users/5aa00739773748469374bdd37bdfc3c2` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:58.476692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014ac4c-c101-49ff-9cb8-43af2254f0ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aa00739773748469374bdd37bdfc3c2)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:58.478608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014ac4c-c101-49ff-9cb8-43af2254f0ea admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:58.529846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7014ac4c-c101-49ff-9cb8-43af2254f0ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aa00739773748469374bdd37bdfc3c2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:58.540380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7014ac4c-c101-49ff-9cb8-43af2254f0ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aa00739773748469374bdd37bdfc3c2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:58.551164 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:58.553347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7014ac4c-c101-49ff-9cb8-43af2254f0ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aa00739773748469374bdd37bdfc3c2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:58.562205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7014ac4c-c101-49ff-9cb8-43af2254f0ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aa00739773748469374bdd37bdfc3c2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:58.563199 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7014ac4c-c101-49ff-9cb8-43af2254f0ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aa00739773748469374bdd37bdfc3c2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:58.569070 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 907/1806] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:58 2022] DELETE /identity/v3/users/5aa00739773748469374bdd37bdfc3c2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:58.579966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d122133-3674-4e99-9815-04f81b6783fd None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:58.580409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d122133-3674-4e99-9815-04f81b6783fd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:58.580950 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d122133-3674-4e99-9815-04f81b6783fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:58.600919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-0d122133-3674-4e99-9815-04f81b6783fd None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:25:58.602130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0d122133-3674-4e99-9815-04f81b6783fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:58.661652 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 900/1807] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:25:58.671440 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014ac4c-c101-49ff-9cb8-43af2254f0ea admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:58.672967 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7014ac4c-c101-49ff-9cb8-43af2254f0ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:58.743120 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014ac4c-c101-49ff-9cb8-43af2254f0ea admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:58.747232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65b267-7364-47cc-ac66-a7ff92a6bc1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:58.747840 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65b267-7364-47cc-ac66-a7ff92a6bc1d service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:58.748117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65b267-7364-47cc-ac66-a7ff92a6bc1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:58.748501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65b267-7364-47cc-ac66-a7ff92a6bc1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:58.758858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65b267-7364-47cc-ac66-a7ff92a6bc1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:58.760326 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65b267-7364-47cc-ac66-a7ff92a6bc1d service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:58.771181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 908/1808] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:25:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:25:59.112594 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d122133-3674-4e99-9815-04f81b6783fd None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:59.114180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0d122133-3674-4e99-9815-04f81b6783fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:59.135141 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d122133-3674-4e99-9815-04f81b6783fd None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:59.140252 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:59.141619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:59.142180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:59.142180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] PATH_INFO: `/v3/projects/987db81b6d1249d0855576f4d3d6e23d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:59.143520 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] RBAC: Authorizing `identity:delete_project(project_id=987db81b6d1249d0855576f4d3d6e23d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:59.145509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:59.187545 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:25:59.196451 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '987db81b6d1249d0855576f4d3d6e23d', 'request_id': 'req-18852d41-606d-416a-8ef3-2b411483890a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.202961 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '987db81b6d1249d0855576f4d3d6e23d', 'request_id': 'req-18852d41-606d-416a-8ef3-2b411483890a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.212301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '987db81b6d1249d0855576f4d3d6e23d', 'request_id': 'req-18852d41-606d-416a-8ef3-2b411483890a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.224129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] The token cache is being invalidate because project 987db81b6d1249d0855576f4d3d6e23d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:59.224443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.225058 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.226548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 901/1809] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:59 2022] DELETE /identity/v3/projects/987db81b6d1249d0855576f4d3d6e23d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:59.230515 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65b267-7364-47cc-ac66-a7ff92a6bc1d service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:59.240406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65b267-7364-47cc-ac66-a7ff92a6bc1d service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:59.300462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc2201-5b7b-4563-9576-7278ed2c6001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:59.301376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc2201-5b7b-4563-9576-7278ed2c6001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:59.301658 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc2201-5b7b-4563-9576-7278ed2c6001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:59.301924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc2201-5b7b-4563-9576-7278ed2c6001 admin admin] PATH_INFO: `/v3/users/3acd7fdb5d8948d0a9761a861a6201c9` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:59.303072 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc2201-5b7b-4563-9576-7278ed2c6001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3acd7fdb5d8948d0a9761a861a6201c9)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:59.305042 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc2201-5b7b-4563-9576-7278ed2c6001 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:59.352981 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-60dc2201-5b7b-4563-9576-7278ed2c6001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3acd7fdb5d8948d0a9761a861a6201c9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.360629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-60dc2201-5b7b-4563-9576-7278ed2c6001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3acd7fdb5d8948d0a9761a861a6201c9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.372929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-60dc2201-5b7b-4563-9576-7278ed2c6001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3acd7fdb5d8948d0a9761a861a6201c9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.380633 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-60dc2201-5b7b-4563-9576-7278ed2c6001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3acd7fdb5d8948d0a9761a861a6201c9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.381273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-60dc2201-5b7b-4563-9576-7278ed2c6001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3acd7fdb5d8948d0a9761a861a6201c9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.387437 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 909/1810] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:59 2022] DELETE /identity/v3/users/3acd7fdb5d8948d0a9761a861a6201c9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:59.738930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:59.740907 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:59.755984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18852d41-606d-416a-8ef3-2b411483890a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:59.817503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:25:59.818047 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:25:59.818322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:25:59.818589 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] PATH_INFO: `/v3/projects/de191e95b74a4a4da6af2318d2d91286` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:25:59.819710 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de191e95b74a4a4da6af2318d2d91286)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:25:59.821595 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:25:59.870114 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de191e95b74a4a4da6af2318d2d91286', 'request_id': 'req-a1a93798-c7a5-4068-a725-3771868d1ed3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.876330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de191e95b74a4a4da6af2318d2d91286', 'request_id': 'req-a1a93798-c7a5-4068-a725-3771868d1ed3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.883141 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de191e95b74a4a4da6af2318d2d91286', 'request_id': 'req-a1a93798-c7a5-4068-a725-3771868d1ed3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.897142 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] The token cache is being invalidate because project de191e95b74a4a4da6af2318d2d91286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:25:59.897438 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.898075 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:25:59.899644 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 902/1811] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:25:59 2022] DELETE /identity/v3/projects/de191e95b74a4a4da6af2318d2d91286 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:25:59.903830 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc2201-5b7b-4563-9576-7278ed2c6001 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:25:59.905439 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-60dc2201-5b7b-4563-9576-7278ed2c6001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:25:59.919054 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc2201-5b7b-4563-9576-7278ed2c6001 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:25:59.923434 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:00.026105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295e9d6-6f80-4af8-8110-865d9f140973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:00.026778 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295e9d6-6f80-4af8-8110-865d9f140973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:00.027059 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295e9d6-6f80-4af8-8110-865d9f140973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:00.027372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295e9d6-6f80-4af8-8110-865d9f140973 admin admin] PATH_INFO: `/v3/users/99eddbeda2db4ad29efabcf6672e8f20` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:00.032694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295e9d6-6f80-4af8-8110-865d9f140973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99eddbeda2db4ad29efabcf6672e8f20)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:00.034914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295e9d6-6f80-4af8-8110-865d9f140973 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:00.043830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:00.045746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:00.074803 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a93798-c7a5-4068-a725-3771868d1ed3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:00.083100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c2220-e295-44d5-86f8-cd0180738dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:00.084191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c2220-e295-44d5-86f8-cd0180738dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:00.084191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c2220-e295-44d5-86f8-cd0180738dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:00.084522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c2220-e295-44d5-86f8-cd0180738dd0 admin admin] PATH_INFO: `/v3/users/b1d3ef1211ee48b3946ae418fb425038` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:00.085983 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c2220-e295-44d5-86f8-cd0180738dd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1d3ef1211ee48b3946ae418fb425038)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:00.088894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c2220-e295-44d5-86f8-cd0180738dd0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:00.121545 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b295e9d6-6f80-4af8-8110-865d9f140973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99eddbeda2db4ad29efabcf6672e8f20'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.132871 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b295e9d6-6f80-4af8-8110-865d9f140973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99eddbeda2db4ad29efabcf6672e8f20'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.159395 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b295e9d6-6f80-4af8-8110-865d9f140973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99eddbeda2db4ad29efabcf6672e8f20'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.166880 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b295e9d6-6f80-4af8-8110-865d9f140973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99eddbeda2db4ad29efabcf6672e8f20'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.166999 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-686c2220-e295-44d5-86f8-cd0180738dd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1d3ef1211ee48b3946ae418fb425038'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.167760 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b295e9d6-6f80-4af8-8110-865d9f140973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99eddbeda2db4ad29efabcf6672e8f20'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.173180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=78,keystone:DELETE=19,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:00.176341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 910/1812] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:25:59 2022] DELETE /identity/v3/users/99eddbeda2db4ad29efabcf6672e8f20 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:00.180724 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-686c2220-e295-44d5-86f8-cd0180738dd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1d3ef1211ee48b3946ae418fb425038'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.181434 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-686c2220-e295-44d5-86f8-cd0180738dd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1d3ef1211ee48b3946ae418fb425038'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.200084 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-686c2220-e295-44d5-86f8-cd0180738dd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1d3ef1211ee48b3946ae418fb425038'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.210705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-686c2220-e295-44d5-86f8-cd0180738dd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1d3ef1211ee48b3946ae418fb425038'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.222561 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 903/1813] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:26:00 2022] DELETE /identity/v3/users/b1d3ef1211ee48b3946ae418fb425038 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:00.228885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb88a0-bcfb-4434-80b7-e35b6cddd9be None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:00.229369 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb88a0-bcfb-4434-80b7-e35b6cddd9be None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:00.230172 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb88a0-bcfb-4434-80b7-e35b6cddd9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:00.253294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-13eb88a0-bcfb-4434-80b7-e35b6cddd9be None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:00.254692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-13eb88a0-bcfb-4434-80b7-e35b6cddd9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:00.337520 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 911/1814] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:00.348627 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c2220-e295-44d5-86f8-cd0180738dd0 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:00.350087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-686c2220-e295-44d5-86f8-cd0180738dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:00.376380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb88a0-bcfb-4434-80b7-e35b6cddd9be None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:00.377806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-13eb88a0-bcfb-4434-80b7-e35b6cddd9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:00.391901 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb88a0-bcfb-4434-80b7-e35b6cddd9be None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:00.396731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a54be8-655e-429d-9b94-15326ee448ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:00.397420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a54be8-655e-429d-9b94-15326ee448ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:00.397699 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a54be8-655e-429d-9b94-15326ee448ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:00.397973 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a54be8-655e-429d-9b94-15326ee448ec admin admin] PATH_INFO: `/v3/users/024e5ba85c6b438380aea5b469f92137` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:00.399146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a54be8-655e-429d-9b94-15326ee448ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=024e5ba85c6b438380aea5b469f92137)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:00.401144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a54be8-655e-429d-9b94-15326ee448ec admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:00.424846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:00.428609 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c2220-e295-44d5-86f8-cd0180738dd0 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:00.433450 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d5ba2-d684-4f8b-b75b-5256c0d23801 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:00.434384 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d5ba2-d684-4f8b-b75b-5256c0d23801 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:00.434578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d5ba2-d684-4f8b-b75b-5256c0d23801 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:00.434826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d5ba2-d684-4f8b-b75b-5256c0d23801 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:00.446749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d5ba2-d684-4f8b-b75b-5256c0d23801 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:00.448233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d5ba2-d684-4f8b-b75b-5256c0d23801 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:00.462010 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 904/1815] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:26:00.462293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-50a54be8-655e-429d-9b94-15326ee448ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024e5ba85c6b438380aea5b469f92137'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.472697 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-50a54be8-655e-429d-9b94-15326ee448ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024e5ba85c6b438380aea5b469f92137'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.491143 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-50a54be8-655e-429d-9b94-15326ee448ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024e5ba85c6b438380aea5b469f92137'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.502282 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-50a54be8-655e-429d-9b94-15326ee448ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024e5ba85c6b438380aea5b469f92137'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.503313 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-50a54be8-655e-429d-9b94-15326ee448ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024e5ba85c6b438380aea5b469f92137'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.509765 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 912/1816] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:26:00 2022] DELETE /identity/v3/users/024e5ba85c6b438380aea5b469f92137 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:26:00.514653 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c798d66-6964-4a4f-b2f6-fec024a9f02a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:00.515012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c798d66-6964-4a4f-b2f6-fec024a9f02a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:00.515385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c798d66-6964-4a4f-b2f6-fec024a9f02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:00.523219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a54be8-655e-429d-9b94-15326ee448ec admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:00.524641 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-50a54be8-655e-429d-9b94-15326ee448ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:00.549538 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7c798d66-6964-4a4f-b2f6-fec024a9f02a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:00.550488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7c798d66-6964-4a4f-b2f6-fec024a9f02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:00.624540 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a54be8-655e-429d-9b94-15326ee448ec admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:00.629506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036745a-ef02-44ce-a182-a17157a09f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:00.629679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 905/1817] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:26:00.630188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036745a-ef02-44ce-a182-a17157a09f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:00.630188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036745a-ef02-44ce-a182-a17157a09f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:00.630404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036745a-ef02-44ce-a182-a17157a09f43 admin admin] PATH_INFO: `/v3/projects/082346d53700453589e502c7c878a679` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:00.631568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036745a-ef02-44ce-a182-a17157a09f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=082346d53700453589e502c7c878a679)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:00.633832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036745a-ef02-44ce-a182-a17157a09f43 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:00.639741 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c798d66-6964-4a4f-b2f6-fec024a9f02a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:00.641366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7c798d66-6964-4a4f-b2f6-fec024a9f02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:00.693356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c036745a-ef02-44ce-a182-a17157a09f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '082346d53700453589e502c7c878a679', 'request_id': 'req-c036745a-ef02-44ce-a182-a17157a09f43'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.699510 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c036745a-ef02-44ce-a182-a17157a09f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '082346d53700453589e502c7c878a679', 'request_id': 'req-c036745a-ef02-44ce-a182-a17157a09f43'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.707132 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c036745a-ef02-44ce-a182-a17157a09f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '082346d53700453589e502c7c878a679', 'request_id': 'req-c036745a-ef02-44ce-a182-a17157a09f43'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.714767 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c798d66-6964-4a4f-b2f6-fec024a9f02a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:00.727801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c036745a-ef02-44ce-a182-a17157a09f43 admin admin] The token cache is being invalidate because project 082346d53700453589e502c7c878a679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:26:00.728190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c036745a-ef02-44ce-a182-a17157a09f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.728949 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c036745a-ef02-44ce-a182-a17157a09f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.731771 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 913/1818] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:26:00 2022] DELETE /identity/v3/projects/082346d53700453589e502c7c878a679 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:00.740286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e5801-9277-441c-9999-1f972ec6875a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:00.740525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e5801-9277-441c-9999-1f972ec6875a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:00.742352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e5801-9277-441c-9999-1f972ec6875a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:00.746432 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:00.762254 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-482e5801-9277-441c-9999-1f972ec6875a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:00.764642 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-482e5801-9277-441c-9999-1f972ec6875a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:00.782610 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975b000-edbf-48d2-8911-c2ed69175391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:00.783426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975b000-edbf-48d2-8911-c2ed69175391 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:00.783698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975b000-edbf-48d2-8911-c2ed69175391 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:00.783978 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975b000-edbf-48d2-8911-c2ed69175391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:00.786154 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f975b000-edbf-48d2-8911-c2ed69175391 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:00.805373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:00.863542 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 914/1819] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:00.864465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e5801-9277-441c-9999-1f972ec6875a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:00.865993 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-482e5801-9277-441c-9999-1f972ec6875a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:00.878590 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e5801-9277-441c-9999-1f972ec6875a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:00.880269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975b000-edbf-48d2-8911-c2ed69175391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:00.881774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975b000-edbf-48d2-8911-c2ed69175391 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:00.883690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242134e-15f9-43f5-aa89-cc37cc3d63e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:00.884145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242134e-15f9-43f5-aa89-cc37cc3d63e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:00.884421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242134e-15f9-43f5-aa89-cc37cc3d63e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:00.884694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242134e-15f9-43f5-aa89-cc37cc3d63e6 admin admin] PATH_INFO: `/v3/projects/ca3c42b1d43f49e3940ff089c2de1bd8` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:00.885847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242134e-15f9-43f5-aa89-cc37cc3d63e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca3c42b1d43f49e3940ff089c2de1bd8)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:00.887796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242134e-15f9-43f5-aa89-cc37cc3d63e6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:00.893012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 906/1820] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:26:00.894943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bcfc6-90ca-43a0-bc60-bd7955c3c513 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:00.895355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bcfc6-90ca-43a0-bc60-bd7955c3c513 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:00.895765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bcfc6-90ca-43a0-bc60-bd7955c3c513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:00.916807 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1c6bcfc6-90ca-43a0-bc60-bd7955c3c513 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:00.918758 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1c6bcfc6-90ca-43a0-bc60-bd7955c3c513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:00.928386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 907/1821] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:00.932836 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bcfc6-90ca-43a0-bc60-bd7955c3c513 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:00.943785 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8242134e-15f9-43f5-aa89-cc37cc3d63e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca3c42b1d43f49e3940ff089c2de1bd8', 'request_id': 'req-8242134e-15f9-43f5-aa89-cc37cc3d63e6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.944277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bcfc6-90ca-43a0-bc60-bd7955c3c513 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:00.949947 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8242134e-15f9-43f5-aa89-cc37cc3d63e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca3c42b1d43f49e3940ff089c2de1bd8', 'request_id': 'req-8242134e-15f9-43f5-aa89-cc37cc3d63e6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.957238 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8242134e-15f9-43f5-aa89-cc37cc3d63e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca3c42b1d43f49e3940ff089c2de1bd8', 'request_id': 'req-8242134e-15f9-43f5-aa89-cc37cc3d63e6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.971671 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8242134e-15f9-43f5-aa89-cc37cc3d63e6 admin admin] The token cache is being invalidate because project ca3c42b1d43f49e3940ff089c2de1bd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:26:00.972098 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8242134e-15f9-43f5-aa89-cc37cc3d63e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.972853 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8242134e-15f9-43f5-aa89-cc37cc3d63e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:00.975036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 915/1822] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:26:00 2022] DELETE /identity/v3/projects/ca3c42b1d43f49e3940ff089c2de1bd8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:00.979545 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ede93f-1efc-4683-9739-4e9f14998a1b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:00.980093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ede93f-1efc-4683-9739-4e9f14998a1b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:00.980372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ede93f-1efc-4683-9739-4e9f14998a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.008068 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-44ede93f-1efc-4683-9739-4e9f14998a1b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:01.009286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-44ede93f-1efc-4683-9739-4e9f14998a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:01.030020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4b076-c01f-40c5-9402-810a51b9f1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:01.030671 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4b076-c01f-40c5-9402-810a51b9f1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.031184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4b076-c01f-40c5-9402-810a51b9f1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.031321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4b076-c01f-40c5-9402-810a51b9f1a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.032538 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4b076-c01f-40c5-9402-810a51b9f1a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:26:01.033112 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4b076-c01f-40c5-9402-810a51b9f1a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:01.035876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4b076-c01f-40c5-9402-810a51b9f1a3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:01.052610 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 908/1823] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:26:01.064554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4b076-c01f-40c5-9402-810a51b9f1a3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:01.068785 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a4e4b076-c01f-40c5-9402-810a51b9f1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:01.094867 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4b076-c01f-40c5-9402-810a51b9f1a3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:01.100110 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b310547-2ec1-4ca1-84fd-ea056c367a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:01.101487 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b310547-2ec1-4ca1-84fd-ea056c367a5c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.101810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b310547-2ec1-4ca1-84fd-ea056c367a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.102260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b310547-2ec1-4ca1-84fd-ea056c367a5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.102929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b310547-2ec1-4ca1-84fd-ea056c367a5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:01.105141 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b310547-2ec1-4ca1-84fd-ea056c367a5c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:01.111712 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 916/1824] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:26:01.120251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06b43c-77e1-4c10-a87e-b81d7bd4b67e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.120613 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06b43c-77e1-4c10-a87e-b81d7bd4b67e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.120972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06b43c-77e1-4c10-a87e-b81d7bd4b67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.138359 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 909/1825] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:26:01 2022] POST /identity/v3/projects => generated 365 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:01.142397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b310547-2ec1-4ca1-84fd-ea056c367a5c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:01.149420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4d06b43c-77e1-4c10-a87e-b81d7bd4b67e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:01.150785 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4d06b43c-77e1-4c10-a87e-b81d7bd4b67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:01.214767 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:01.223155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:01.240795 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b310547-2ec1-4ca1-84fd-ea056c367a5c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:01.244572 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 917/1826] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:26:01.245956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06b43c-77e1-4c10-a87e-b81d7bd4b67e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:01.248275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5a441-c955-4943-93af-edd46872486c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:01.248910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5a441-c955-4943-93af-edd46872486c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.249187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5a441-c955-4943-93af-edd46872486c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.249456 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5a441-c955-4943-93af-edd46872486c admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.250641 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf5a441-c955-4943-93af-edd46872486c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:01.253681 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf5a441-c955-4943-93af-edd46872486c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:01.257145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06b43c-77e1-4c10-a87e-b81d7bd4b67e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:01.261796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0198e-7a98-4d39-87e6-25286556ff4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:01.262467 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0198e-7a98-4d39-87e6-25286556ff4c admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.263051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0198e-7a98-4d39-87e6-25286556ff4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.263545 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0198e-7a98-4d39-87e6-25286556ff4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.264227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0198e-7a98-4d39-87e6-25286556ff4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:26:01.264540 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0198e-7a98-4d39-87e6-25286556ff4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:01.266256 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0198e-7a98-4d39-87e6-25286556ff4c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:01.283107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 910/1827] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:26:01 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:26:01.283296 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 918/1828] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:26:01.286169 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0198e-7a98-4d39-87e6-25286556ff4c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:01.286288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5a441-c955-4943-93af-edd46872486c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:01.296747 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0198e-7a98-4d39-87e6-25286556ff4c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:01.297045 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5a441-c955-4943-93af-edd46872486c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:01.301667 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8127db3-5929-40ec-abd0-787df4cc705a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:01.302090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877567a-4501-4f40-b9d9-bdb6a557f51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:01.302279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8127db3-5929-40ec-abd0-787df4cc705a admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.302558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8127db3-5929-40ec-abd0-787df4cc705a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.302705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877567a-4501-4f40-b9d9-bdb6a557f51b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.302813 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8127db3-5929-40ec-abd0-787df4cc705a admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.302968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877567a-4501-4f40-b9d9-bdb6a557f51b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.303389 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877567a-4501-4f40-b9d9-bdb6a557f51b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.303483 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8127db3-5929-40ec-abd0-787df4cc705a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:01.304028 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877567a-4501-4f40-b9d9-bdb6a557f51b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:01.305378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8127db3-5929-40ec-abd0-787df4cc705a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:01.305619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877567a-4501-4f40-b9d9-bdb6a557f51b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:01.321978 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 911/1829] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:01.323048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877567a-4501-4f40-b9d9-bdb6a557f51b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:01.333855 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877567a-4501-4f40-b9d9-bdb6a557f51b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:01.339864 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dadb00-9cd4-49bb-9588-6ca4a5517608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:01.340480 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dadb00-9cd4-49bb-9588-6ca4a5517608 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.340762 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dadb00-9cd4-49bb-9588-6ca4a5517608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.341037 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dadb00-9cd4-49bb-9588-6ca4a5517608 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.341948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dadb00-9cd4-49bb-9588-6ca4a5517608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:01.342053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 919/1830] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:26:01 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:01.342626 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8127db3-5929-40ec-abd0-787df4cc705a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:01.343363 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dadb00-9cd4-49bb-9588-6ca4a5517608 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:01.345543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c8127db3-5929-40ec-abd0-787df4cc705a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:01.357201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 912/1831] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:26:01.357794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dadb00-9cd4-49bb-9588-6ca4a5517608 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:01.368881 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dadb00-9cd4-49bb-9588-6ca4a5517608 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:01.428677 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8127db3-5929-40ec-abd0-787df4cc705a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:01.433341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:01.435628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.435628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.435628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] PATH_INFO: `/v3/projects/4f63d365cfff4ffe8156ff5ef7f58839` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.437402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f63d365cfff4ffe8156ff5ef7f58839)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:01.439943 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:01.440124 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e5262-fc78-4e9c-896c-729bf9d01261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:01.440761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e5262-fc78-4e9c-896c-729bf9d01261 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.441044 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e5262-fc78-4e9c-896c-729bf9d01261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.441316 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e5262-fc78-4e9c-896c-729bf9d01261 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.441950 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e5262-fc78-4e9c-896c-729bf9d01261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:01.443908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e5262-fc78-4e9c-896c-729bf9d01261 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:01.467830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 913/1832] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:26:01 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:01.469408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e5262-fc78-4e9c-896c-729bf9d01261 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:01.479405 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e5262-fc78-4e9c-896c-729bf9d01261 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:01.501899 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f63d365cfff4ffe8156ff5ef7f58839', 'request_id': 'req-234809c5-fedb-4907-adbc-43abd63f25ce'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:01.508703 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f63d365cfff4ffe8156ff5ef7f58839', 'request_id': 'req-234809c5-fedb-4907-adbc-43abd63f25ce'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:01.517990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f63d365cfff4ffe8156ff5ef7f58839', 'request_id': 'req-234809c5-fedb-4907-adbc-43abd63f25ce'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:01.535814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] The token cache is being invalidate because project 4f63d365cfff4ffe8156ff5ef7f58839 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:26:01.536245 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:01.537025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:01.539386 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 920/1833] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:26:01 2022] DELETE /identity/v3/projects/4f63d365cfff4ffe8156ff5ef7f58839 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:01.539703 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:01.541444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:01.559164 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84eedcd-da1e-4d84-83d5-a36fa2053cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:01.559970 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84eedcd-da1e-4d84-83d5-a36fa2053cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.560873 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84eedcd-da1e-4d84-83d5-a36fa2053cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.561255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84eedcd-da1e-4d84-83d5-a36fa2053cdd admin admin] PATH_INFO: `/v3/projects/2435041277d7474d9adc98df66e1c8f3/users/21f5d29434e349af9e49bf5c53d33a70/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.581612 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84eedcd-da1e-4d84-83d5-a36fa2053cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2435041277d7474d9adc98df66e1c8f3, user_id=21f5d29434e349af9e49bf5c53d33a70, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:01.584570 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84eedcd-da1e-4d84-83d5-a36fa2053cdd admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:01.607155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 914/1834] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:01 2022] PUT /identity/v3/projects/2435041277d7474d9adc98df66e1c8f3/users/21f5d29434e349af9e49bf5c53d33a70/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:01.611054 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795b528-b25e-4e6f-8070-a534bd8acd5f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.611519 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795b528-b25e-4e6f-8070-a534bd8acd5f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.611930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795b528-b25e-4e6f-8070-a534bd8acd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.618809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:01.627691 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234809c5-fedb-4907-adbc-43abd63f25ce admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:01.647112 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c795b528-b25e-4e6f-8070-a534bd8acd5f None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:01.648593 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c795b528-b25e-4e6f-8070-a534bd8acd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:01.672912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:01.727991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c696a0d-aca7-46b6-bd0c-ff6b8fc1dcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:01.727991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c696a0d-aca7-46b6-bd0c-ff6b8fc1dcba admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.727991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c696a0d-aca7-46b6-bd0c-ff6b8fc1dcba admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.727991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c696a0d-aca7-46b6-bd0c-ff6b8fc1dcba admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.728408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c696a0d-aca7-46b6-bd0c-ff6b8fc1dcba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:01.728408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c696a0d-aca7-46b6-bd0c-ff6b8fc1dcba admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:01.741980 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 921/1835] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:01.747517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e3573-fa05-457a-b870-f0dc8a8888cf None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.747809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e3573-fa05-457a-b870-f0dc8a8888cf None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.748087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e3573-fa05-457a-b870-f0dc8a8888cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.770703 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 915/1836] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:01.777568 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795b528-b25e-4e6f-8070-a534bd8acd5f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:01.789662 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795b528-b25e-4e6f-8070-a534bd8acd5f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:01.791563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c04e3573-fa05-457a-b870-f0dc8a8888cf None None] MFA Rules not processed for user `21f5d29434e349af9e49bf5c53d33a70`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:01.792794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c04e3573-fa05-457a-b870-f0dc8a8888cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:01.795139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c2bad-d9a9-4e02-9d59-17470119343e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:01.795778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c2bad-d9a9-4e02-9d59-17470119343e admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.796060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c2bad-d9a9-4e02-9d59-17470119343e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.796334 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c2bad-d9a9-4e02-9d59-17470119343e admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.797517 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c2bad-d9a9-4e02-9d59-17470119343e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:01.799168 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c2bad-d9a9-4e02-9d59-17470119343e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:01.810616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 916/1837] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:01.811990 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d405c-c0cf-46d8-972f-77877995c103 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.812282 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d405c-c0cf-46d8-972f-77877995c103 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.812616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d405c-c0cf-46d8-972f-77877995c103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.835387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-595d405c-c0cf-46d8-972f-77877995c103 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:01.836619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-595d405c-c0cf-46d8-972f-77877995c103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:01.848802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 917/1838] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:01.852105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d405c-c0cf-46d8-972f-77877995c103 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:01.861141 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d405c-c0cf-46d8-972f-77877995c103 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:01.866643 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9792063-f027-4d31-8752-56730ca3f5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:01.868555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9792063-f027-4d31-8752-56730ca3f5b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.868845 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9792063-f027-4d31-8752-56730ca3f5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.869126 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9792063-f027-4d31-8752-56730ca3f5b4 admin admin] PATH_INFO: `/v3/projects/ace6580a1307486895a0ec8149a95fa9/users/5e7267cf9bbf4bea81bc949221f31e18/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.878704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 922/1839] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:01.880170 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e3573-fa05-457a-b870-f0dc8a8888cf None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:01.890236 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9792063-f027-4d31-8752-56730ca3f5b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ace6580a1307486895a0ec8149a95fa9, user_id=5e7267cf9bbf4bea81bc949221f31e18, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:01.892730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e3573-fa05-457a-b870-f0dc8a8888cf None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:01.893675 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9792063-f027-4d31-8752-56730ca3f5b4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:01.897645 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330c44a-b00e-4a6a-aa69-474b5700dfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:01.898284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330c44a-b00e-4a6a-aa69-474b5700dfcc admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.898528 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330c44a-b00e-4a6a-aa69-474b5700dfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.898798 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330c44a-b00e-4a6a-aa69-474b5700dfcc admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.899544 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330c44a-b00e-4a6a-aa69-474b5700dfcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:26:01.899885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330c44a-b00e-4a6a-aa69-474b5700dfcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:01.901547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330c44a-b00e-4a6a-aa69-474b5700dfcc admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:01.911975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 918/1840] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:01 2022] PUT /identity/v3/projects/ace6580a1307486895a0ec8149a95fa9/users/5e7267cf9bbf4bea81bc949221f31e18/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:01.915563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9792063-f027-4d31-8752-56730ca3f5b4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:01.915900 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b9792063-f027-4d31-8752-56730ca3f5b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:01.925682 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 923/1841] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:26:01.928477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81176d8-2d5c-4134-99b9-bb5d4a77076d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:01.928856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81176d8-2d5c-4134-99b9-bb5d4a77076d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:01.929140 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81176d8-2d5c-4134-99b9-bb5d4a77076d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:01.948733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d81176d8-2d5c-4134-99b9-bb5d4a77076d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:01.949974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d81176d8-2d5c-4134-99b9-bb5d4a77076d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:01.997367 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9792063-f027-4d31-8752-56730ca3f5b4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:02.003529 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41545435-62c0-4f08-ac48-0a1daf8d21aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:02.004298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41545435-62c0-4f08-ac48-0a1daf8d21aa admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.004833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41545435-62c0-4f08-ac48-0a1daf8d21aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.005177 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41545435-62c0-4f08-ac48-0a1daf8d21aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.006302 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41545435-62c0-4f08-ac48-0a1daf8d21aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:02.008292 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41545435-62c0-4f08-ac48-0a1daf8d21aa admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:02.021824 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 924/1842] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:02.022661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81176d8-2d5c-4134-99b9-bb5d4a77076d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:02.035439 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81176d8-2d5c-4134-99b9-bb5d4a77076d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:02.041288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd42a3c-59c8-4f5e-910b-fb0bef6e1a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:02.042241 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd42a3c-59c8-4f5e-910b-fb0bef6e1a04 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.042694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd42a3c-59c8-4f5e-910b-fb0bef6e1a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.043119 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd42a3c-59c8-4f5e-910b-fb0bef6e1a04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.043845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd42a3c-59c8-4f5e-910b-fb0bef6e1a04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:02.049263 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 919/1843] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:26:01 2022] POST /identity/v3/projects => generated 365 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:02.049471 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd42a3c-59c8-4f5e-910b-fb0bef6e1a04 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:02.049585 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41545435-62c0-4f08-ac48-0a1daf8d21aa admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:02.050944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-41545435-62c0-4f08-ac48-0a1daf8d21aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:02.053379 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:02.087474 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 925/1844] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:26:02 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:02.090281 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd42a3c-59c8-4f5e-910b-fb0bef6e1a04 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:02.132806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41545435-62c0-4f08-ac48-0a1daf8d21aa admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:02.155528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:02.171210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd42a3c-59c8-4f5e-910b-fb0bef6e1a04 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:02.179361 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88149e54-487c-43bd-aa1c-fba3ff4fcdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:02.180305 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88149e54-487c-43bd-aa1c-fba3ff4fcdfd admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.180883 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88149e54-487c-43bd-aa1c-fba3ff4fcdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.181001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88149e54-487c-43bd-aa1c-fba3ff4fcdfd admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.181622 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88149e54-487c-43bd-aa1c-fba3ff4fcdfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:02.184474 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88149e54-487c-43bd-aa1c-fba3ff4fcdfd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:02.189453 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb3061-a1be-4903-a571-e6111a36129e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:02.190308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb3061-a1be-4903-a571-e6111a36129e service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.190736 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb3061-a1be-4903-a571-e6111a36129e service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.191203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb3061-a1be-4903-a571-e6111a36129e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.202525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb3061-a1be-4903-a571-e6111a36129e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:02.204616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb3061-a1be-4903-a571-e6111a36129e service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:02.208085 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 926/1845] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:26:02 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:02.209021 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88149e54-487c-43bd-aa1c-fba3ff4fcdfd admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:02.218435 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 920/1846] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:26:02.220218 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb3061-a1be-4903-a571-e6111a36129e service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:02.228462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88149e54-487c-43bd-aa1c-fba3ff4fcdfd admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:02.232366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb3061-a1be-4903-a571-e6111a36129e service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:02.235375 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb70eac-ace2-4958-a3e3-b777c20f7714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:02.236011 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb70eac-ace2-4958-a3e3-b777c20f7714 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.236289 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb70eac-ace2-4958-a3e3-b777c20f7714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.236803 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb70eac-ace2-4958-a3e3-b777c20f7714 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.237601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb70eac-ace2-4958-a3e3-b777c20f7714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:02.240542 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f966e6-d7a7-4e81-b706-44b080627b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:02.241138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb70eac-ace2-4958-a3e3-b777c20f7714 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:02.242008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f966e6-d7a7-4e81-b706-44b080627b8f admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.242929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f966e6-d7a7-4e81-b706-44b080627b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.243466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f966e6-d7a7-4e81-b706-44b080627b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.244610 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f966e6-d7a7-4e81-b706-44b080627b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:02.246806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f966e6-d7a7-4e81-b706-44b080627b8f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:02.260083 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 921/1847] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:02.266203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f966e6-d7a7-4e81-b706-44b080627b8f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:02.280615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f966e6-d7a7-4e81-b706-44b080627b8f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:02.284113 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 927/1848] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:26:02 2022] POST /identity/v3/users => generated 286 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:02.288182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb70eac-ace2-4958-a3e3-b777c20f7714 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:02.289885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee5038-5fac-4088-8785-9edf678b8082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:02.290816 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee5038-5fac-4088-8785-9edf678b8082 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.291919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee5038-5fac-4088-8785-9edf678b8082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.305099 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb70eac-ace2-4958-a3e3-b777c20f7714 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:02.313222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4425b-16bb-45fa-b789-42ffe7f0f81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:02.313632 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee5038-5fac-4088-8785-9edf678b8082 admin admin] PATH_INFO: `/v3/projects/016da3213a6440f2a79791e77dd826ae/users/26c89ff39c6b43f2a922d8843a30d2b6/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.314678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4425b-16bb-45fa-b789-42ffe7f0f81b admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.315234 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4425b-16bb-45fa-b789-42ffe7f0f81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.315417 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4425b-16bb-45fa-b789-42ffe7f0f81b admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.317093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4425b-16bb-45fa-b789-42ffe7f0f81b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:02.319129 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4425b-16bb-45fa-b789-42ffe7f0f81b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:02.338312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 928/1849] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:26:02.344148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4425b-16bb-45fa-b789-42ffe7f0f81b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:02.355089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee5038-5fac-4088-8785-9edf678b8082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=016da3213a6440f2a79791e77dd826ae, user_id=26c89ff39c6b43f2a922d8843a30d2b6, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:02.358043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee5038-5fac-4088-8785-9edf678b8082 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:02.360039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4425b-16bb-45fa-b789-42ffe7f0f81b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:02.366681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8cd4d-3a9e-4263-b2f3-b1a2b5b7c204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:02.367481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8cd4d-3a9e-4263-b2f3-b1a2b5b7c204 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.367761 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8cd4d-3a9e-4263-b2f3-b1a2b5b7c204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.368045 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8cd4d-3a9e-4263-b2f3-b1a2b5b7c204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.369890 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8cd4d-3a9e-4263-b2f3-b1a2b5b7c204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:02.373623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8cd4d-3a9e-4263-b2f3-b1a2b5b7c204 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:02.391514 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 929/1851] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:02.392716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 922/1851] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:02 2022] PUT /identity/v3/projects/016da3213a6440f2a79791e77dd826ae/users/26c89ff39c6b43f2a922d8843a30d2b6/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:26:02.393722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8cd4d-3a9e-4263-b2f3-b1a2b5b7c204 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:02.395306 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee5038-5fac-4088-8785-9edf678b8082 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:02.409183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8cd4d-3a9e-4263-b2f3-b1a2b5b7c204 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:02.512041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8fa89-4c47-4c78-b2f3-c2444cc3037a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:02.512041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8fa89-4c47-4c78-b2f3-c2444cc3037a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.512041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8fa89-4c47-4c78-b2f3-c2444cc3037a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.512041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8fa89-4c47-4c78-b2f3-c2444cc3037a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.512440 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8fa89-4c47-4c78-b2f3-c2444cc3037a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:02.518396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8fa89-4c47-4c78-b2f3-c2444cc3037a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:02.524793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee5038-5fac-4088-8785-9edf678b8082 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:02.534165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79467b-a09d-4dcf-989f-e46bfc7745ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:02.535699 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79467b-a09d-4dcf-989f-e46bfc7745ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.536163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79467b-a09d-4dcf-989f-e46bfc7745ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.536269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 930/1852] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:02.536714 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79467b-a09d-4dcf-989f-e46bfc7745ea admin admin] PATH_INFO: `/v3/projects/f2a51798d7e54605a7e59b0cba385973/users/d40a56908e934174b1f7b20e62e2cfcd/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.537142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8fa89-4c47-4c78-b2f3-c2444cc3037a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:02.551659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8fa89-4c47-4c78-b2f3-c2444cc3037a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:02.557029 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fc9c9-cd0b-4b83-bf79-36c3fc90e978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:02.557722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fc9c9-cd0b-4b83-bf79-36c3fc90e978 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.558180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fc9c9-cd0b-4b83-bf79-36c3fc90e978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.558465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fc9c9-cd0b-4b83-bf79-36c3fc90e978 admin admin] PATH_INFO: `/v3/projects/016da3213a6440f2a79791e77dd826ae/users/26c89ff39c6b43f2a922d8843a30d2b6/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.560961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92fc9c9-cd0b-4b83-bf79-36c3fc90e978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=016da3213a6440f2a79791e77dd826ae, user_id=26c89ff39c6b43f2a922d8843a30d2b6, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:02.562735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79467b-a09d-4dcf-989f-e46bfc7745ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a51798d7e54605a7e59b0cba385973, user_id=d40a56908e934174b1f7b20e62e2cfcd, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:02.564404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92fc9c9-cd0b-4b83-bf79-36c3fc90e978 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:02.567153 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79467b-a09d-4dcf-989f-e46bfc7745ea admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:02.583280 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 931/1853] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:02 2022] PUT /identity/v3/projects/016da3213a6440f2a79791e77dd826ae/users/26c89ff39c6b43f2a922d8843a30d2b6/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:02.586827 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 923/1854] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:02 2022] PUT /identity/v3/projects/f2a51798d7e54605a7e59b0cba385973/users/d40a56908e934174b1f7b20e62e2cfcd/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:02.594394 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6de0d-6955-440b-875d-64016f6981dc None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.594870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6de0d-6955-440b-875d-64016f6981dc None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.595217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6de0d-6955-440b-875d-64016f6981dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.597682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8c089-9a0f-4e52-8911-485315e5a8ba None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.598206 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8c089-9a0f-4e52-8911-485315e5a8ba None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.598586 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8c089-9a0f-4e52-8911-485315e5a8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.625675 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f8a6de0d-6955-440b-875d-64016f6981dc None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:02.627400 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f8a6de0d-6955-440b-875d-64016f6981dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:02.643617 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-86d8c089-9a0f-4e52-8911-485315e5a8ba None None] MFA Rules not processed for user `26c89ff39c6b43f2a922d8843a30d2b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:02.644780 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:02.647001 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-86d8c089-9a0f-4e52-8911-485315e5a8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:02.728013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 932/1855] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:02.740796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6de0d-6955-440b-875d-64016f6981dc None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:02.758183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:02.764697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 924/1856] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:02.778524 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8c089-9a0f-4e52-8911-485315e5a8ba None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:02.780239 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-86d8c089-9a0f-4e52-8911-485315e5a8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:02.806688 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6de0d-6955-440b-875d-64016f6981dc None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:02.810674 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca6cc1-4116-4b9f-bb98-15c4c190af99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:02.811318 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca6cc1-4116-4b9f-bb98-15c4c190af99 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.811597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca6cc1-4116-4b9f-bb98-15c4c190af99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.811866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca6cc1-4116-4b9f-bb98-15c4c190af99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.838971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca6cc1-4116-4b9f-bb98-15c4c190af99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:02.840537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca6cc1-4116-4b9f-bb98-15c4c190af99 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:02.851549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 933/1857] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:26:02.869866 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8c089-9a0f-4e52-8911-485315e5a8ba None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:02.875802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114db78-2634-4d19-b17d-7a5b9203014c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:02.876928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114db78-2634-4d19-b17d-7a5b9203014c service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:02.877180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114db78-2634-4d19-b17d-7a5b9203014c service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:02.877919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114db78-2634-4d19-b17d-7a5b9203014c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:02.887609 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114db78-2634-4d19-b17d-7a5b9203014c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:02.889085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114db78-2634-4d19-b17d-7a5b9203014c service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:02.899758 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 925/1858] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:26:02.997185 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca6cc1-4116-4b9f-bb98-15c4c190af99 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:03.012229 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca6cc1-4116-4b9f-bb98-15c4c190af99 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:03.017905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b9b0d-5631-4168-a003-7d71458b79d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:03.019251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b9b0d-5631-4168-a003-7d71458b79d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:03.019697 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b9b0d-5631-4168-a003-7d71458b79d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:03.020122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b9b0d-5631-4168-a003-7d71458b79d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:03.092422 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b9b0d-5631-4168-a003-7d71458b79d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:03.094074 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b9b0d-5631-4168-a003-7d71458b79d5 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:03.105100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 934/1859] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:26:07.954426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114db78-2634-4d19-b17d-7a5b9203014c service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:07.956572 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a114db78-2634-4d19-b17d-7a5b9203014c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:07.973733 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114db78-2634-4d19-b17d-7a5b9203014c service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:07.978848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29630abd-5cc5-4b3e-a30d-585949d2b29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:07.979564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29630abd-5cc5-4b3e-a30d-585949d2b29e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:07.980016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29630abd-5cc5-4b3e-a30d-585949d2b29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:07.980515 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29630abd-5cc5-4b3e-a30d-585949d2b29e admin admin] PATH_INFO: `/v3/users/bad955c1488c4c3380db7e84403253ee` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:07.981790 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29630abd-5cc5-4b3e-a30d-585949d2b29e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad955c1488c4c3380db7e84403253ee)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:07.984197 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29630abd-5cc5-4b3e-a30d-585949d2b29e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:08.081349 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-29630abd-5cc5-4b3e-a30d-585949d2b29e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad955c1488c4c3380db7e84403253ee'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:08.107042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-29630abd-5cc5-4b3e-a30d-585949d2b29e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad955c1488c4c3380db7e84403253ee'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:08.107042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-29630abd-5cc5-4b3e-a30d-585949d2b29e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad955c1488c4c3380db7e84403253ee'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:08.136050 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-29630abd-5cc5-4b3e-a30d-585949d2b29e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad955c1488c4c3380db7e84403253ee'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:08.153358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-29630abd-5cc5-4b3e-a30d-585949d2b29e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad955c1488c4c3380db7e84403253ee'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:08.165689 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 926/1860] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:26:07 2022] DELETE /identity/v3/users/bad955c1488c4c3380db7e84403253ee => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:08.625420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b9b0d-5631-4168-a003-7d71458b79d5 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:08.643824 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b9b0d-5631-4168-a003-7d71458b79d5 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:08.780350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af2eb0-7142-45d8-8e2a-0a7a967659c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:08.781706 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af2eb0-7142-45d8-8e2a-0a7a967659c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:08.781987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af2eb0-7142-45d8-8e2a-0a7a967659c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:08.782253 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af2eb0-7142-45d8-8e2a-0a7a967659c4 admin admin] PATH_INFO: `/v3/projects/1a37a13b40e54342bb3942ff4387534e` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:08.783695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af2eb0-7142-45d8-8e2a-0a7a967659c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a37a13b40e54342bb3942ff4387534e)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:08.785936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af2eb0-7142-45d8-8e2a-0a7a967659c4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:08.789419 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:08.855200 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-03af2eb0-7142-45d8-8e2a-0a7a967659c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a37a13b40e54342bb3942ff4387534e', 'request_id': 'req-03af2eb0-7142-45d8-8e2a-0a7a967659c4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:08.863711 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-03af2eb0-7142-45d8-8e2a-0a7a967659c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a37a13b40e54342bb3942ff4387534e', 'request_id': 'req-03af2eb0-7142-45d8-8e2a-0a7a967659c4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:08.872494 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-03af2eb0-7142-45d8-8e2a-0a7a967659c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a37a13b40e54342bb3942ff4387534e', 'request_id': 'req-03af2eb0-7142-45d8-8e2a-0a7a967659c4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:08.892028 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-03af2eb0-7142-45d8-8e2a-0a7a967659c4 admin admin] The token cache is being invalidate because project 1a37a13b40e54342bb3942ff4387534e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:26:08.892368 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-03af2eb0-7142-45d8-8e2a-0a7a967659c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:08.892925 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-03af2eb0-7142-45d8-8e2a-0a7a967659c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:08.895120 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 935/1861] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:26:08 2022] DELETE /identity/v3/projects/1a37a13b40e54342bb3942ff4387534e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:08.900769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9f94a-ae69-449e-a63d-57c150a24a6c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:08.901925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9f94a-ae69-449e-a63d-57c150a24a6c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:08.902355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9f94a-ae69-449e-a63d-57c150a24a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:08.932048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-aae9f94a-ae69-449e-a63d-57c150a24a6c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:08.934189 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-aae9f94a-ae69-449e-a63d-57c150a24a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:09.070924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 927/1862] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:09.078762 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be27528-ed40-4255-a4d8-0d93bb068f39 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:09.079206 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be27528-ed40-4255-a4d8-0d93bb068f39 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:09.079577 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be27528-ed40-4255-a4d8-0d93bb068f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:09.116339 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-7be27528-ed40-4255-a4d8-0d93bb068f39 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:09.118257 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7be27528-ed40-4255-a4d8-0d93bb068f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:09.137032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 936/1863] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:09.141713 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9f94a-ae69-449e-a63d-57c150a24a6c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:09.162351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9f94a-ae69-449e-a63d-57c150a24a6c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:09.167855 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca97cd-9f92-4d5d-86d0-3975468c98f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:09.170095 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca97cd-9f92-4d5d-86d0-3975468c98f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:09.170505 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca97cd-9f92-4d5d-86d0-3975468c98f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:09.171018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca97cd-9f92-4d5d-86d0-3975468c98f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:09.172264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca97cd-9f92-4d5d-86d0-3975468c98f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:26:09.172569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca97cd-9f92-4d5d-86d0-3975468c98f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:09.176502 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca97cd-9f92-4d5d-86d0-3975468c98f5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:09.194201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 928/1864] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:26:09.201344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be27528-ed40-4255-a4d8-0d93bb068f39 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:09.213250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be27528-ed40-4255-a4d8-0d93bb068f39 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:09.254384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac601836-47fa-4fd6-891e-ecb788305aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:09.255695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac601836-47fa-4fd6-891e-ecb788305aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:09.256251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac601836-47fa-4fd6-891e-ecb788305aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:09.267888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac601836-47fa-4fd6-891e-ecb788305aa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:09.268611 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac601836-47fa-4fd6-891e-ecb788305aa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:09.273035 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac601836-47fa-4fd6-891e-ecb788305aa5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:09.399310 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca97cd-9f92-4d5d-86d0-3975468c98f5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:09.400063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 937/1865] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:26:09 2022] POST /identity/v3/projects => generated 375 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:09.404825 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:09.420977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca97cd-9f92-4d5d-86d0-3975468c98f5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:09.502060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac601836-47fa-4fd6-891e-ecb788305aa5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:09.503907 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ac601836-47fa-4fd6-891e-ecb788305aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:09.608103 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f7a12-f54a-4a50-9a5d-1ff88b55914d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:09.610050 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f7a12-f54a-4a50-9a5d-1ff88b55914d admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:09.616520 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f7a12-f54a-4a50-9a5d-1ff88b55914d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:09.616703 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f7a12-f54a-4a50-9a5d-1ff88b55914d admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:09.617841 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f7a12-f54a-4a50-9a5d-1ff88b55914d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:09.622296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f7a12-f54a-4a50-9a5d-1ff88b55914d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:09.678738 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac601836-47fa-4fd6-891e-ecb788305aa5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:09.704593 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c513ab-9236-4d3a-8e87-7a4b7dd96bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:09.705623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c513ab-9236-4d3a-8e87-7a4b7dd96bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:09.706063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c513ab-9236-4d3a-8e87-7a4b7dd96bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:09.706334 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c513ab-9236-4d3a-8e87-7a4b7dd96bf3 admin admin] PATH_INFO: `/v3/users/21f5d29434e349af9e49bf5c53d33a70` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:09.711629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c513ab-9236-4d3a-8e87-7a4b7dd96bf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21f5d29434e349af9e49bf5c53d33a70)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:09.713635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c513ab-9236-4d3a-8e87-7a4b7dd96bf3 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:09.719330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 929/1866] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:26:09 2022] POST /identity/v3/users => generated 288 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:09.744986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f7a12-f54a-4a50-9a5d-1ff88b55914d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:09.772657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f7a12-f54a-4a50-9a5d-1ff88b55914d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:09.782043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac011e92-9783-4f2e-9807-973ac9a9dab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:09.785164 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac011e92-9783-4f2e-9807-973ac9a9dab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:09.785496 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac011e92-9783-4f2e-9807-973ac9a9dab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:09.785774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac011e92-9783-4f2e-9807-973ac9a9dab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:09.786466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac011e92-9783-4f2e-9807-973ac9a9dab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:09.789148 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac011e92-9783-4f2e-9807-973ac9a9dab2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:09.807447 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 930/1867] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:09.808758 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac011e92-9783-4f2e-9807-973ac9a9dab2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:09.826749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac011e92-9783-4f2e-9807-973ac9a9dab2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:09.833649 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01276b42-d649-45f6-884b-28ffd213ce52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:09.834294 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01276b42-d649-45f6-884b-28ffd213ce52 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:09.834617 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01276b42-d649-45f6-884b-28ffd213ce52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:09.834897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01276b42-d649-45f6-884b-28ffd213ce52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:09.835722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01276b42-d649-45f6-884b-28ffd213ce52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:09.838639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01276b42-d649-45f6-884b-28ffd213ce52 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:09.871773 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 931/1868] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:09.877279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01276b42-d649-45f6-884b-28ffd213ce52 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:09.891268 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01276b42-d649-45f6-884b-28ffd213ce52 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:09.912015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4cb0b-e7d7-4000-a032-b504a5d64d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:09.912719 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4cb0b-e7d7-4000-a032-b504a5d64d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:09.913000 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4cb0b-e7d7-4000-a032-b504a5d64d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:09.913330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4cb0b-e7d7-4000-a032-b504a5d64d28 admin admin] PATH_INFO: `/v3/projects/f6ff3da9d38f423ba8fbd11f635f2e1e/users/d9f9821cd90e4d9096cc01aee5c0e569/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:09.957847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-47c513ab-9236-4d3a-8e87-7a4b7dd96bf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21f5d29434e349af9e49bf5c53d33a70'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:09.958896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4cb0b-e7d7-4000-a032-b504a5d64d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ff3da9d38f423ba8fbd11f635f2e1e, user_id=d9f9821cd90e4d9096cc01aee5c0e569, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:09.964014 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4cb0b-e7d7-4000-a032-b504a5d64d28 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:09.975422 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-47c513ab-9236-4d3a-8e87-7a4b7dd96bf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21f5d29434e349af9e49bf5c53d33a70'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:09.987579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:10.004680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 932/1869] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:09 2022] PUT /identity/v3/projects/f6ff3da9d38f423ba8fbd11f635f2e1e/users/d9f9821cd90e4d9096cc01aee5c0e569/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:26:10.014085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f4626-0751-4c6d-bfcd-f8d06a2f32dd None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:10.015258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f4626-0751-4c6d-bfcd-f8d06a2f32dd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:10.015790 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f4626-0751-4c6d-bfcd-f8d06a2f32dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:10.021593 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-47c513ab-9236-4d3a-8e87-7a4b7dd96bf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21f5d29434e349af9e49bf5c53d33a70'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:10.032749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-47c513ab-9236-4d3a-8e87-7a4b7dd96bf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21f5d29434e349af9e49bf5c53d33a70'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:10.033664 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-47c513ab-9236-4d3a-8e87-7a4b7dd96bf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21f5d29434e349af9e49bf5c53d33a70'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:10.040595 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-246f4626-0751-4c6d-bfcd-f8d06a2f32dd None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:10.040832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 938/1870] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:26:09 2022] DELETE /identity/v3/users/21f5d29434e349af9e49bf5c53d33a70 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:10.042285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-246f4626-0751-4c6d-bfcd-f8d06a2f32dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:10.045347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7e8e8-1a94-4bfb-9de7-00b3f897c970 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:10.045635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7e8e8-1a94-4bfb-9de7-00b3f897c970 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:10.045914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7e8e8-1a94-4bfb-9de7-00b3f897c970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:10.105755 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-92a7e8e8-1a94-4bfb-9de7-00b3f897c970 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:10.107087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-92a7e8e8-1a94-4bfb-9de7-00b3f897c970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:10.126598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 933/1871] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:10.135008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f4626-0751-4c6d-bfcd-f8d06a2f32dd None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:10.139046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-246f4626-0751-4c6d-bfcd-f8d06a2f32dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:10.207207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 939/1872] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:10.213137 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7e8e8-1a94-4bfb-9de7-00b3f897c970 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:10.214598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-92a7e8e8-1a94-4bfb-9de7-00b3f897c970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:10.272170 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:10.300208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f4626-0751-4c6d-bfcd-f8d06a2f32dd None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:10.324814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36669868-683d-43ae-8c44-cefd1af40cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:10.325461 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36669868-683d-43ae-8c44-cefd1af40cdd service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:10.331437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36669868-683d-43ae-8c44-cefd1af40cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:10.332125 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36669868-683d-43ae-8c44-cefd1af40cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:10.359590 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36669868-683d-43ae-8c44-cefd1af40cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:10.361169 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36669868-683d-43ae-8c44-cefd1af40cdd service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:10.388482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7e8e8-1a94-4bfb-9de7-00b3f897c970 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:10.415104 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4a98c-342c-489f-9c3c-62626ecfa21a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:10.415860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4a98c-342c-489f-9c3c-62626ecfa21a service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:10.416140 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4a98c-342c-489f-9c3c-62626ecfa21a service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:10.416421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4a98c-342c-489f-9c3c-62626ecfa21a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:10.433919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 934/1873] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:26:10.474766 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4a98c-342c-489f-9c3c-62626ecfa21a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:10.479936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4a98c-342c-489f-9c3c-62626ecfa21a service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:10.527440 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 940/1874] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:26:11.775561 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36669868-683d-43ae-8c44-cefd1af40cdd service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:11.783578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-36669868-683d-43ae-8c44-cefd1af40cdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:11.820755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36669868-683d-43ae-8c44-cefd1af40cdd service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:11.837879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:11.840424 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:11.842161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:11.842550 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] PATH_INFO: `/v3/projects/2435041277d7474d9adc98df66e1c8f3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:11.848156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2435041277d7474d9adc98df66e1c8f3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:11.850708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:11.902843 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357f95d-6db0-458c-bf8f-ad61e9445034 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:11.903458 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357f95d-6db0-458c-bf8f-ad61e9445034 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:11.903575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357f95d-6db0-458c-bf8f-ad61e9445034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:12.020633 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5357f95d-6db0-458c-bf8f-ad61e9445034 None None] MFA Rules not processed for user `5e7267cf9bbf4bea81bc949221f31e18`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:12.022122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5357f95d-6db0-458c-bf8f-ad61e9445034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:12.071195 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2435041277d7474d9adc98df66e1c8f3', 'request_id': 'req-11a4e24c-04c2-47b5-adbc-338130136a8c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:12.094750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2435041277d7474d9adc98df66e1c8f3', 'request_id': 'req-11a4e24c-04c2-47b5-adbc-338130136a8c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:12.123864 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2435041277d7474d9adc98df66e1c8f3', 'request_id': 'req-11a4e24c-04c2-47b5-adbc-338130136a8c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:12.173585 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] The token cache is being invalidate because project 2435041277d7474d9adc98df66e1c8f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:26:12.173804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:12.177051 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:12.182035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 935/1875] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:26:11 2022] DELETE /identity/v3/projects/2435041277d7474d9adc98df66e1c8f3 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:12.186427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:12.192435 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:12.292108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 941/1876] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:26:12.296878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357f95d-6db0-458c-bf8f-ad61e9445034 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:12.300597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5357f95d-6db0-458c-bf8f-ad61e9445034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:12.380391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:12.440334 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4e24c-04c2-47b5-adbc-338130136a8c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:12.470993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47336dc0-2c2e-4c2c-92e9-6c3dfaf7af23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:12.472057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47336dc0-2c2e-4c2c-92e9-6c3dfaf7af23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:12.472340 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47336dc0-2c2e-4c2c-92e9-6c3dfaf7af23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:12.473216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47336dc0-2c2e-4c2c-92e9-6c3dfaf7af23 admin admin] PATH_INFO: `/v3/users/26c89ff39c6b43f2a922d8843a30d2b6` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:12.477079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47336dc0-2c2e-4c2c-92e9-6c3dfaf7af23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c89ff39c6b43f2a922d8843a30d2b6)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:12.479204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47336dc0-2c2e-4c2c-92e9-6c3dfaf7af23 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:12.536462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357f95d-6db0-458c-bf8f-ad61e9445034 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:12.543948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d967f2-1db9-443b-a7e0-fb713b2dca74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:12.544591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d967f2-1db9-443b-a7e0-fb713b2dca74 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:12.544879 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d967f2-1db9-443b-a7e0-fb713b2dca74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:12.552533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d967f2-1db9-443b-a7e0-fb713b2dca74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:12.552533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d967f2-1db9-443b-a7e0-fb713b2dca74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:12.582797 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d967f2-1db9-443b-a7e0-fb713b2dca74 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:12.671978 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-47336dc0-2c2e-4c2c-92e9-6c3dfaf7af23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c89ff39c6b43f2a922d8843a30d2b6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:12.709504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-47336dc0-2c2e-4c2c-92e9-6c3dfaf7af23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c89ff39c6b43f2a922d8843a30d2b6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:12.710397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-47336dc0-2c2e-4c2c-92e9-6c3dfaf7af23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c89ff39c6b43f2a922d8843a30d2b6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:12.713471 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 942/1877] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:26:12 2022] POST /identity/v3/projects => generated 385 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:12.717965 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d967f2-1db9-443b-a7e0-fb713b2dca74 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:12.720529 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f3d967f2-1db9-443b-a7e0-fb713b2dca74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:12.761213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-47336dc0-2c2e-4c2c-92e9-6c3dfaf7af23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c89ff39c6b43f2a922d8843a30d2b6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:12.771541 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:12.800376 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-47336dc0-2c2e-4c2c-92e9-6c3dfaf7af23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c89ff39c6b43f2a922d8843a30d2b6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:12.821516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 936/1878] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:26:12 2022] DELETE /identity/v3/users/26c89ff39c6b43f2a922d8843a30d2b6 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:12.998527 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d967f2-1db9-443b-a7e0-fb713b2dca74 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:13.053941 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa461a-8601-4488-8221-1081b4b45ebb None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:13.058724 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa461a-8601-4488-8221-1081b4b45ebb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:13.059053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa461a-8601-4488-8221-1081b4b45ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:13.136905 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-cdaa461a-8601-4488-8221-1081b4b45ebb None None] MFA Rules not processed for user `d40a56908e934174b1f7b20e62e2cfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:13.138157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cdaa461a-8601-4488-8221-1081b4b45ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:13.227915 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f0029-b67c-429a-b5b3-04a0fc42a9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:13.228532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f0029-b67c-429a-b5b3-04a0fc42a9ae admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:13.228814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f0029-b67c-429a-b5b3-04a0fc42a9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:13.229166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f0029-b67c-429a-b5b3-04a0fc42a9ae admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:13.229730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f0029-b67c-429a-b5b3-04a0fc42a9ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:13.232057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f0029-b67c-429a-b5b3-04a0fc42a9ae admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:13.287484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 943/1879] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:26:12 2022] POST /identity/v3/users => generated 299 bytes in 573 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:13.291529 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f0029-b67c-429a-b5b3-04a0fc42a9ae admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:13.315241 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f0029-b67c-429a-b5b3-04a0fc42a9ae admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:13.315241 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1137b2f-9188-4a0e-a390-497d9270c23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:13.315241 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1137b2f-9188-4a0e-a390-497d9270c23f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:13.315241 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1137b2f-9188-4a0e-a390-497d9270c23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:13.315601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1137b2f-9188-4a0e-a390-497d9270c23f admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:13.315601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1137b2f-9188-4a0e-a390-497d9270c23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:13.315841 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1137b2f-9188-4a0e-a390-497d9270c23f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:13.341823 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 937/1880] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:26:13.348655 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa461a-8601-4488-8221-1081b4b45ebb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:13.351500 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 944/1881] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:26:13.352656 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1137b2f-9188-4a0e-a390-497d9270c23f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:13.359653 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cdaa461a-8601-4488-8221-1081b4b45ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:13.378185 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1137b2f-9188-4a0e-a390-497d9270c23f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:13.388819 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5bcfa9-4e70-4371-98ac-1ce3ef3f1abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:13.392490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5bcfa9-4e70-4371-98ac-1ce3ef3f1abd admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:13.393792 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5bcfa9-4e70-4371-98ac-1ce3ef3f1abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:13.396245 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5bcfa9-4e70-4371-98ac-1ce3ef3f1abd admin admin] PATH_INFO: `/v3/projects/45d54683464248b08141b7fe35047ecb/users/6fa96892d57d4d969e01d9014a50a8e9/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:13.408539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa461a-8601-4488-8221-1081b4b45ebb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:13.413657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4aba6-2176-4c70-a5a8-d1d61babe4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:13.414301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4aba6-2176-4c70-a5a8-d1d61babe4fc admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:13.414577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4aba6-2176-4c70-a5a8-d1d61babe4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:13.414840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4aba6-2176-4c70-a5a8-d1d61babe4fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:13.415639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4aba6-2176-4c70-a5a8-d1d61babe4fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:13.417601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4aba6-2176-4c70-a5a8-d1d61babe4fc admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:13.448324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5bcfa9-4e70-4371-98ac-1ce3ef3f1abd admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d54683464248b08141b7fe35047ecb, user_id=6fa96892d57d4d969e01d9014a50a8e9, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:13.455044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5bcfa9-4e70-4371-98ac-1ce3ef3f1abd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:13.492607 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 938/1882] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:26:13 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:13.501879 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 945/1883] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:13 2022] PUT /identity/v3/projects/45d54683464248b08141b7fe35047ecb/users/6fa96892d57d4d969e01d9014a50a8e9/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:13.502106 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4aba6-2176-4c70-a5a8-d1d61babe4fc admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:13.505561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5bcfa9-4e70-4371-98ac-1ce3ef3f1abd admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:13.524276 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:13.675972 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4aba6-2176-4c70-a5a8-d1d61babe4fc admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:13.679285 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5bcfa9-4e70-4371-98ac-1ce3ef3f1abd admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:13.694455 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3003769-3f52-4dcf-a349-425a76b73ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:13.695030 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3003769-3f52-4dcf-a349-425a76b73ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:13.695339 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3003769-3f52-4dcf-a349-425a76b73ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:13.695614 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3003769-3f52-4dcf-a349-425a76b73ce2 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:13.696265 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3003769-3f52-4dcf-a349-425a76b73ce2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:13.698704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3003769-3f52-4dcf-a349-425a76b73ce2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:13.702205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a1203-f026-4504-81ee-3fba73aba6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:13.702885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a1203-f026-4504-81ee-3fba73aba6ee admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:13.703189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a1203-f026-4504-81ee-3fba73aba6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:13.703462 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a1203-f026-4504-81ee-3fba73aba6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:13.704121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a1203-f026-4504-81ee-3fba73aba6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:13.705732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a1203-f026-4504-81ee-3fba73aba6ee admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:13.747764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 946/1884] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:13.752858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a1203-f026-4504-81ee-3fba73aba6ee admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:13.788215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 939/1885] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:26:13 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:13.789578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3003769-3f52-4dcf-a349-425a76b73ce2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:13.794746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a1203-f026-4504-81ee-3fba73aba6ee admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:13.797401 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b3003769-3f52-4dcf-a349-425a76b73ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:13.809069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a841ca-991a-4337-8214-b1a8bc40f0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:13.809948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a841ca-991a-4337-8214-b1a8bc40f0a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:13.810064 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a841ca-991a-4337-8214-b1a8bc40f0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:13.810287 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a841ca-991a-4337-8214-b1a8bc40f0a7 admin admin] PATH_INFO: `/v3/projects/45d54683464248b08141b7fe35047ecb/users/6fa96892d57d4d969e01d9014a50a8e9/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:13.813880 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a841ca-991a-4337-8214-b1a8bc40f0a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d54683464248b08141b7fe35047ecb, user_id=6fa96892d57d4d969e01d9014a50a8e9, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:13.818374 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a841ca-991a-4337-8214-b1a8bc40f0a7 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:13.823804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3003769-3f52-4dcf-a349-425a76b73ce2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:13.833524 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a9236-bbd6-4ba0-802b-bf235cb7e66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:13.834293 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9236-bbd6-4ba0-802b-bf235cb7e66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:13.834577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9236-bbd6-4ba0-802b-bf235cb7e66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:13.834851 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9236-bbd6-4ba0-802b-bf235cb7e66d admin admin] PATH_INFO: `/v3/projects/016da3213a6440f2a79791e77dd826ae` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:13.836083 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a9236-bbd6-4ba0-802b-bf235cb7e66d admin admin] RBAC: Authorizing `identity:delete_project(project_id=016da3213a6440f2a79791e77dd826ae)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:13.838031 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a9236-bbd6-4ba0-802b-bf235cb7e66d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:13.849946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 947/1886] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:13 2022] PUT /identity/v3/projects/45d54683464248b08141b7fe35047ecb/users/6fa96892d57d4d969e01d9014a50a8e9/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:13.850969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a841ca-991a-4337-8214-b1a8bc40f0a7 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:13.966664 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:13.983107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-973a9236-bbd6-4ba0-802b-bf235cb7e66d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '016da3213a6440f2a79791e77dd826ae', 'request_id': 'req-973a9236-bbd6-4ba0-802b-bf235cb7e66d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:13.998454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-973a9236-bbd6-4ba0-802b-bf235cb7e66d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '016da3213a6440f2a79791e77dd826ae', 'request_id': 'req-973a9236-bbd6-4ba0-802b-bf235cb7e66d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:14.019808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-973a9236-bbd6-4ba0-802b-bf235cb7e66d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '016da3213a6440f2a79791e77dd826ae', 'request_id': 'req-973a9236-bbd6-4ba0-802b-bf235cb7e66d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:14.033367 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a841ca-991a-4337-8214-b1a8bc40f0a7 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:14.042599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-973a9236-bbd6-4ba0-802b-bf235cb7e66d admin admin] The token cache is being invalidate because project 016da3213a6440f2a79791e77dd826ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:26:14.043035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-973a9236-bbd6-4ba0-802b-bf235cb7e66d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:14.045130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-973a9236-bbd6-4ba0-802b-bf235cb7e66d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:14.050284 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 940/1887] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:26:13 2022] DELETE /identity/v3/projects/016da3213a6440f2a79791e77dd826ae => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:14.055344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b6251-76e5-4e03-a2b5-f0d2986a8992 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:14.055344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b6251-76e5-4e03-a2b5-f0d2986a8992 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:14.056062 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b6251-76e5-4e03-a2b5-f0d2986a8992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:14.110064 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7d6b6251-76e5-4e03-a2b5-f0d2986a8992 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:14.114426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7d6b6251-76e5-4e03-a2b5-f0d2986a8992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:14.217902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe276e1-33e6-4ea0-addf-8b9fac9d67f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:14.219858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe276e1-33e6-4ea0-addf-8b9fac9d67f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:14.221275 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe276e1-33e6-4ea0-addf-8b9fac9d67f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:14.221567 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe276e1-33e6-4ea0-addf-8b9fac9d67f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:14.223431 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe276e1-33e6-4ea0-addf-8b9fac9d67f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:14.229111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe276e1-33e6-4ea0-addf-8b9fac9d67f5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:14.258224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 948/1888] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:14.260248 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe276e1-33e6-4ea0-addf-8b9fac9d67f5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:14.268215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ffe276e1-33e6-4ea0-addf-8b9fac9d67f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:14.307915 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe276e1-33e6-4ea0-addf-8b9fac9d67f5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:14.317711 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 941/1889] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:26:14.328488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e97a5-5ef0-4103-8e94-8872caf0aebd None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:14.328711 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb839c03-8fb3-42b5-ac69-c212d971a7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:14.328865 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e97a5-5ef0-4103-8e94-8872caf0aebd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:14.329037 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e97a5-5ef0-4103-8e94-8872caf0aebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:14.329406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb839c03-8fb3-42b5-ac69-c212d971a7c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:14.329689 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb839c03-8fb3-42b5-ac69-c212d971a7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:14.332128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb839c03-8fb3-42b5-ac69-c212d971a7c8 admin admin] PATH_INFO: `/v3/projects/a85ca09a00f045af8bccae7383271502/users/e7e25970bb534ea39821f9c9640dcb66/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:14.372392 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb839c03-8fb3-42b5-ac69-c212d971a7c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85ca09a00f045af8bccae7383271502, user_id=e7e25970bb534ea39821f9c9640dcb66, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:14.376401 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb839c03-8fb3-42b5-ac69-c212d971a7c8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:14.409094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-0e0e97a5-5ef0-4103-8e94-8872caf0aebd None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:14.410386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0e0e97a5-5ef0-4103-8e94-8872caf0aebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:14.436364 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 949/1890] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:14 2022] PUT /identity/v3/projects/a85ca09a00f045af8bccae7383271502/users/e7e25970bb534ea39821f9c9640dcb66/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:26:14.440258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb839c03-8fb3-42b5-ac69-c212d971a7c8 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:14.448426 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:14.580650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb839c03-8fb3-42b5-ac69-c212d971a7c8 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:14.589918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a39af3-5926-4f57-bd79-9e9eceaab1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:14.590842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a39af3-5926-4f57-bd79-9e9eceaab1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:14.591360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a39af3-5926-4f57-bd79-9e9eceaab1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:14.591785 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a39af3-5926-4f57-bd79-9e9eceaab1c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:14.593279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 942/1891] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:14.599961 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e97a5-5ef0-4103-8e94-8872caf0aebd None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:14.608265 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a39af3-5926-4f57-bd79-9e9eceaab1c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:14.611858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a39af3-5926-4f57-bd79-9e9eceaab1c3 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:14.625387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e97a5-5ef0-4103-8e94-8872caf0aebd None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:14.650765 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 950/1892] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:14.652333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef312d-098a-4f5b-85e6-7e18a2e1b501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:14.652822 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a39af3-5926-4f57-bd79-9e9eceaab1c3 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:14.662601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef312d-098a-4f5b-85e6-7e18a2e1b501 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:14.663021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef312d-098a-4f5b-85e6-7e18a2e1b501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:14.663356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef312d-098a-4f5b-85e6-7e18a2e1b501 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:14.664294 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef312d-098a-4f5b-85e6-7e18a2e1b501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:26:14.664610 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef312d-098a-4f5b-85e6-7e18a2e1b501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:14.673165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef312d-098a-4f5b-85e6-7e18a2e1b501 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:14.677321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a39af3-5926-4f57-bd79-9e9eceaab1c3 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:14.688103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96095d04-326b-490b-add1-952345859fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:14.688803 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96095d04-326b-490b-add1-952345859fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:14.689165 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96095d04-326b-490b-add1-952345859fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:14.689348 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96095d04-326b-490b-add1-952345859fc8 admin admin] PATH_INFO: `/v3/projects/a85ca09a00f045af8bccae7383271502/users/e7e25970bb534ea39821f9c9640dcb66/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:14.697058 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96095d04-326b-490b-add1-952345859fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85ca09a00f045af8bccae7383271502, user_id=e7e25970bb534ea39821f9c9640dcb66, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:14.700894 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96095d04-326b-490b-add1-952345859fc8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:14.702828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 943/1893] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:26:14.703970 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef312d-098a-4f5b-85e6-7e18a2e1b501 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:14.716365 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef312d-098a-4f5b-85e6-7e18a2e1b501 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:14.749741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 951/1894] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:14 2022] PUT /identity/v3/projects/a85ca09a00f045af8bccae7383271502/users/e7e25970bb534ea39821f9c9640dcb66/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:14.861934 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d0069-bd0f-427c-a842-10b573d15543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:14.862737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d0069-bd0f-427c-a842-10b573d15543 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:14.863022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d0069-bd0f-427c-a842-10b573d15543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:14.863320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d0069-bd0f-427c-a842-10b573d15543 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:14.863974 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d0069-bd0f-427c-a842-10b573d15543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:14.866183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d0069-bd0f-427c-a842-10b573d15543 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:14.899651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:15.014715 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 944/1895] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:26:14 2022] POST /identity/v3/projects => generated 381 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:15.019734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96095d04-326b-490b-add1-952345859fc8 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:15.049288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96095d04-326b-490b-add1-952345859fc8 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:15.201225 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907e8fa-fa79-4f75-b538-33ac6cac3f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:15.201914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907e8fa-fa79-4f75-b538-33ac6cac3f68 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:15.202291 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907e8fa-fa79-4f75-b538-33ac6cac3f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:15.202616 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907e8fa-fa79-4f75-b538-33ac6cac3f68 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:15.203748 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907e8fa-fa79-4f75-b538-33ac6cac3f68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:15.212483 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907e8fa-fa79-4f75-b538-33ac6cac3f68 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:15.251697 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 952/1896] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:26:15 2022] POST /identity/v3/users => generated 291 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:15.254658 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d0069-bd0f-427c-a842-10b573d15543 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:15.274790 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d0069-bd0f-427c-a842-10b573d15543 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:15.289553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed113b-881b-4f71-9239-a5a3c5c562f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:15.291350 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed113b-881b-4f71-9239-a5a3c5c562f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:15.291710 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed113b-881b-4f71-9239-a5a3c5c562f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:15.292314 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed113b-881b-4f71-9239-a5a3c5c562f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:15.293097 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed113b-881b-4f71-9239-a5a3c5c562f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:15.300423 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed113b-881b-4f71-9239-a5a3c5c562f9 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:15.385193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 945/1897] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:15.389081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907e8fa-fa79-4f75-b538-33ac6cac3f68 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:15.401787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907e8fa-fa79-4f75-b538-33ac6cac3f68 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:15.413688 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9d8b4-1e04-4b8b-bbaf-893878b66c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:15.414375 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9d8b4-1e04-4b8b-bbaf-893878b66c8d admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:15.414729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9d8b4-1e04-4b8b-bbaf-893878b66c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:15.415085 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9d8b4-1e04-4b8b-bbaf-893878b66c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:15.415857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9d8b4-1e04-4b8b-bbaf-893878b66c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:15.419201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9d8b4-1e04-4b8b-bbaf-893878b66c8d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:15.449936 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed113b-881b-4f71-9239-a5a3c5c562f9 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:15.450473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 953/1898] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:26:15.462162 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed113b-881b-4f71-9239-a5a3c5c562f9 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:15.481945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8ed5b-2a8e-432e-a8d9-fdc17e736aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:15.482862 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8ed5b-2a8e-432e-a8d9-fdc17e736aed admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:15.483373 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8ed5b-2a8e-432e-a8d9-fdc17e736aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:15.484654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8ed5b-2a8e-432e-a8d9-fdc17e736aed admin admin] PATH_INFO: `/v3/projects/05bd812ced7549459f75fce518b233a5/users/96324ebf717544c1b2dc6a4b34028548/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:15.525249 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8ed5b-2a8e-432e-a8d9-fdc17e736aed admin admin] RBAC: Authorizing `identity:create_grant(project_id=05bd812ced7549459f75fce518b233a5, user_id=96324ebf717544c1b2dc6a4b34028548, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:15.528144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8ed5b-2a8e-432e-a8d9-fdc17e736aed admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:15.589232 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 946/1899] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:15 2022] PUT /identity/v3/projects/05bd812ced7549459f75fce518b233a5/users/96324ebf717544c1b2dc6a4b34028548/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:15.601573 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0868-41ad-4dff-9a62-88affbb71160 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:15.601995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0868-41ad-4dff-9a62-88affbb71160 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:15.602461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0868-41ad-4dff-9a62-88affbb71160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:15.680384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bedf0868-41ad-4dff-9a62-88affbb71160 None None] MFA Rules not processed for user `96324ebf717544c1b2dc6a4b34028548`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:15.683589 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bedf0868-41ad-4dff-9a62-88affbb71160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:15.856293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:15.878507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 954/1900] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:15.883780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8ed5b-2a8e-432e-a8d9-fdc17e736aed admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:16.113189 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8ed5b-2a8e-432e-a8d9-fdc17e736aed admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:16.118479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcb01a-ed63-44da-8c11-02b7c9e78d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:16.119318 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcb01a-ed63-44da-8c11-02b7c9e78d74 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:16.119587 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcb01a-ed63-44da-8c11-02b7c9e78d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:16.119861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcb01a-ed63-44da-8c11-02b7c9e78d74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:16.120570 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdcb01a-ed63-44da-8c11-02b7c9e78d74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:16.122879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdcb01a-ed63-44da-8c11-02b7c9e78d74 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:16.170889 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 947/1901] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:26:15 2022] POST /identity/v3/projects => generated 383 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:16.175700 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf0868-41ad-4dff-9a62-88affbb71160 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:16.194025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf0868-41ad-4dff-9a62-88affbb71160 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:16.316166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9240d-eb14-4455-8d22-d97db024e4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:16.316956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9240d-eb14-4455-8d22-d97db024e4ba admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:16.317460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9240d-eb14-4455-8d22-d97db024e4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:16.317856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9240d-eb14-4455-8d22-d97db024e4ba admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:16.318601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9240d-eb14-4455-8d22-d97db024e4ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:16.320737 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9240d-eb14-4455-8d22-d97db024e4ba admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:16.349255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 955/1902] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:26:16 2022] POST /identity/v3/users => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:16.353575 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcb01a-ed63-44da-8c11-02b7c9e78d74 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:16.370569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcb01a-ed63-44da-8c11-02b7c9e78d74 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:16.386217 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8edfd4-3371-48fb-ab62-8e33f620b3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:16.386998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8edfd4-3371-48fb-ab62-8e33f620b3ef admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:16.388223 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8edfd4-3371-48fb-ab62-8e33f620b3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:16.388571 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8edfd4-3371-48fb-ab62-8e33f620b3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:16.389354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8edfd4-3371-48fb-ab62-8e33f620b3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:16.391528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8edfd4-3371-48fb-ab62-8e33f620b3ef admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:16.433155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 948/1903] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:16.437426 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9240d-eb14-4455-8d22-d97db024e4ba admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:16.452236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9240d-eb14-4455-8d22-d97db024e4ba admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:16.459150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770f68e-da8e-4d0e-bbe1-59496f8a877a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:16.459944 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770f68e-da8e-4d0e-bbe1-59496f8a877a admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:16.460223 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770f68e-da8e-4d0e-bbe1-59496f8a877a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:16.460496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770f68e-da8e-4d0e-bbe1-59496f8a877a admin admin] PATH_INFO: `/v3/projects/f971461bcef54bb5a7de9216db3925ce/users/af30e0414efd4ce68dc3bb483ee5cd45/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:16.504299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770f68e-da8e-4d0e-bbe1-59496f8a877a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f971461bcef54bb5a7de9216db3925ce, user_id=af30e0414efd4ce68dc3bb483ee5cd45, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:16.516224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770f68e-da8e-4d0e-bbe1-59496f8a877a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:16.555315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 956/1904] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:16 2022] PUT /identity/v3/projects/f971461bcef54bb5a7de9216db3925ce/users/af30e0414efd4ce68dc3bb483ee5cd45/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:16.559475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8edfd4-3371-48fb-ab62-8e33f620b3ef admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:16.573657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8edfd4-3371-48fb-ab62-8e33f620b3ef admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:16.704342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54550c-2eed-4415-9bef-e06a1e877066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:16.704934 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54550c-2eed-4415-9bef-e06a1e877066 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:16.705601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54550c-2eed-4415-9bef-e06a1e877066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:16.705874 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54550c-2eed-4415-9bef-e06a1e877066 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:16.706913 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d54550c-2eed-4415-9bef-e06a1e877066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:16.708669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d54550c-2eed-4415-9bef-e06a1e877066 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:16.770811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 949/1905] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:16.772214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770f68e-da8e-4d0e-bbe1-59496f8a877a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:16.793265 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770f68e-da8e-4d0e-bbe1-59496f8a877a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:16.809169 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb10bd2-b24e-42c4-a872-66720ebe224c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:16.811020 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb10bd2-b24e-42c4-a872-66720ebe224c admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:16.811330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb10bd2-b24e-42c4-a872-66720ebe224c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:16.811606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb10bd2-b24e-42c4-a872-66720ebe224c admin admin] PATH_INFO: `/v3/projects/f971461bcef54bb5a7de9216db3925ce/users/af30e0414efd4ce68dc3bb483ee5cd45/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:16.817657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb10bd2-b24e-42c4-a872-66720ebe224c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f971461bcef54bb5a7de9216db3925ce, user_id=af30e0414efd4ce68dc3bb483ee5cd45, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:16.822740 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb10bd2-b24e-42c4-a872-66720ebe224c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:16.873997 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 957/1906] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:16 2022] PUT /identity/v3/projects/f971461bcef54bb5a7de9216db3925ce/users/af30e0414efd4ce68dc3bb483ee5cd45/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:16.876602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c56b-5f2f-41fc-96b5-be15eb66b159 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:16.877199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c56b-5f2f-41fc-96b5-be15eb66b159 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:16.878403 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c56b-5f2f-41fc-96b5-be15eb66b159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:16.950085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f808c56b-5f2f-41fc-96b5-be15eb66b159 None None] MFA Rules not processed for user `af30e0414efd4ce68dc3bb483ee5cd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:16.952292 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f808c56b-5f2f-41fc-96b5-be15eb66b159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:16.975315 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:17.183757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 950/1907] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:26:17.199160 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb10bd2-b24e-42c4-a872-66720ebe224c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:17.201491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8fb10bd2-b24e-42c4-a872-66720ebe224c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:17.358242 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb10bd2-b24e-42c4-a872-66720ebe224c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:17.375155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485b237-aea4-4e2c-9f63-de0178047d8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:17.377207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485b237-aea4-4e2c-9f63-de0178047d8b service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:17.377207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485b237-aea4-4e2c-9f63-de0178047d8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:17.377207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485b237-aea4-4e2c-9f63-de0178047d8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:17.407262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485b237-aea4-4e2c-9f63-de0178047d8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:17.408634 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485b237-aea4-4e2c-9f63-de0178047d8b service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:17.440985 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 958/1908] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:26:17.621598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808c56b-5f2f-41fc-96b5-be15eb66b159 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:17.633361 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808c56b-5f2f-41fc-96b5-be15eb66b159 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:17.651749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6c6a6-4067-4b06-b7ee-7516794d77f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:17.652394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6c6a6-4067-4b06-b7ee-7516794d77f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:17.652676 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6c6a6-4067-4b06-b7ee-7516794d77f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:17.652951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6c6a6-4067-4b06-b7ee-7516794d77f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:17.805823 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6c6a6-4067-4b06-b7ee-7516794d77f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:17.807324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6c6a6-4067-4b06-b7ee-7516794d77f9 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:17.830090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 951/1909] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:26:18.406118 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485b237-aea4-4e2c-9f63-de0178047d8b service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:18.547615 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:18.575964 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485b237-aea4-4e2c-9f63-de0178047d8b service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:18.586292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b8c9d-fb54-404e-bb91-bdc875a115fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:18.587125 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b8c9d-fb54-404e-bb91-bdc875a115fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:18.587521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b8c9d-fb54-404e-bb91-bdc875a115fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:18.587922 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b8c9d-fb54-404e-bb91-bdc875a115fb admin admin] PATH_INFO: `/v3/users/96324ebf717544c1b2dc6a4b34028548` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:18.594455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b8c9d-fb54-404e-bb91-bdc875a115fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=96324ebf717544c1b2dc6a4b34028548)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:18.596688 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b8c9d-fb54-404e-bb91-bdc875a115fb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:18.728832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bb8b8c9d-fb54-404e-bb91-bdc875a115fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96324ebf717544c1b2dc6a4b34028548'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:18.752067 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bb8b8c9d-fb54-404e-bb91-bdc875a115fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96324ebf717544c1b2dc6a4b34028548'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:18.783303 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bb8b8c9d-fb54-404e-bb91-bdc875a115fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96324ebf717544c1b2dc6a4b34028548'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:18.798304 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bb8b8c9d-fb54-404e-bb91-bdc875a115fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96324ebf717544c1b2dc6a4b34028548'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:18.799388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bb8b8c9d-fb54-404e-bb91-bdc875a115fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96324ebf717544c1b2dc6a4b34028548'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:18.812861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 959/1910] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:26:18 2022] DELETE /identity/v3/users/96324ebf717544c1b2dc6a4b34028548 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:18.818137 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e3cd6-224d-42fe-9d60-a5f4ae2da8f0 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:18.818650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e3cd6-224d-42fe-9d60-a5f4ae2da8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:18.818746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e3cd6-224d-42fe-9d60-a5f4ae2da8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:18.879788 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4f9e3cd6-224d-42fe-9d60-a5f4ae2da8f0 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:18.881401 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4f9e3cd6-224d-42fe-9d60-a5f4ae2da8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:19.026840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 952/1911] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:19.034075 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b8c9d-fb54-404e-bb91-bdc875a115fb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:19.035713 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bb8b8c9d-fb54-404e-bb91-bdc875a115fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:19.145779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b8c9d-fb54-404e-bb91-bdc875a115fb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:19.150324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2e121-a785-4fe6-b8be-29d5dbaf8ffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:19.151056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2e121-a785-4fe6-b8be-29d5dbaf8ffa service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:19.151514 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2e121-a785-4fe6-b8be-29d5dbaf8ffa service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:19.151952 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2e121-a785-4fe6-b8be-29d5dbaf8ffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:19.180929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2e121-a785-4fe6-b8be-29d5dbaf8ffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:19.184150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2e121-a785-4fe6-b8be-29d5dbaf8ffa service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:19.214870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 960/1912] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:26:19.997076 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e3cd6-224d-42fe-9d60-a5f4ae2da8f0 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:19.998564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4f9e3cd6-224d-42fe-9d60-a5f4ae2da8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:20.023446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e3cd6-224d-42fe-9d60-a5f4ae2da8f0 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:20.031391 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:20.032311 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:20.032948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:20.033420 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] PATH_INFO: `/v3/projects/05bd812ced7549459f75fce518b233a5` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:20.035208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] RBAC: Authorizing `identity:delete_project(project_id=05bd812ced7549459f75fce518b233a5)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:20.038254 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:20.104964 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:20.113481 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05bd812ced7549459f75fce518b233a5', 'request_id': 'req-30e6db5e-4459-495a-91b8-5537011a735a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:20.122945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05bd812ced7549459f75fce518b233a5', 'request_id': 'req-30e6db5e-4459-495a-91b8-5537011a735a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:20.134623 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05bd812ced7549459f75fce518b233a5', 'request_id': 'req-30e6db5e-4459-495a-91b8-5537011a735a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:20.160376 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] The token cache is being invalidate because project 05bd812ced7549459f75fce518b233a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:26:20.160677 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:20.162157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:20.166380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 953/1913] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:26:19 2022] DELETE /identity/v3/projects/05bd812ced7549459f75fce518b233a5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:20.170284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2e121-a785-4fe6-b8be-29d5dbaf8ffa service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:20.185899 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2e121-a785-4fe6-b8be-29d5dbaf8ffa service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:20.321494 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a226b-c2a6-42a0-bee9-96e4e64bd5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:20.322167 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a226b-c2a6-42a0-bee9-96e4e64bd5d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:20.322456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a226b-c2a6-42a0-bee9-96e4e64bd5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:20.322735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a226b-c2a6-42a0-bee9-96e4e64bd5d4 admin admin] PATH_INFO: `/v3/users/af30e0414efd4ce68dc3bb483ee5cd45` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:20.324146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a226b-c2a6-42a0-bee9-96e4e64bd5d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af30e0414efd4ce68dc3bb483ee5cd45)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:20.326104 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a226b-c2a6-42a0-bee9-96e4e64bd5d4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:20.349213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:20.421532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-323a226b-c2a6-42a0-bee9-96e4e64bd5d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af30e0414efd4ce68dc3bb483ee5cd45'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:20.435284 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-323a226b-c2a6-42a0-bee9-96e4e64bd5d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af30e0414efd4ce68dc3bb483ee5cd45'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:20.463578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-323a226b-c2a6-42a0-bee9-96e4e64bd5d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af30e0414efd4ce68dc3bb483ee5cd45'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:20.480455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-323a226b-c2a6-42a0-bee9-96e4e64bd5d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af30e0414efd4ce68dc3bb483ee5cd45'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:20.482026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-323a226b-c2a6-42a0-bee9-96e4e64bd5d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af30e0414efd4ce68dc3bb483ee5cd45'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:20.497658 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 961/1914] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:26:20 2022] DELETE /identity/v3/users/af30e0414efd4ce68dc3bb483ee5cd45 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:21.417244 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:21.420470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:21.437887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6db5e-4459-495a-91b8-5537011a735a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:21.566037 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49437ecc-797c-4c68-9a74-7614a49e4edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:21.569029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49437ecc-797c-4c68-9a74-7614a49e4edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:21.569727 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49437ecc-797c-4c68-9a74-7614a49e4edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:21.570665 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49437ecc-797c-4c68-9a74-7614a49e4edd admin admin] PATH_INFO: `/v3/projects/f971461bcef54bb5a7de9216db3925ce` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:21.573203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49437ecc-797c-4c68-9a74-7614a49e4edd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f971461bcef54bb5a7de9216db3925ce)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:21.578794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49437ecc-797c-4c68-9a74-7614a49e4edd admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:21.660078 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-49437ecc-797c-4c68-9a74-7614a49e4edd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f971461bcef54bb5a7de9216db3925ce', 'request_id': 'req-49437ecc-797c-4c68-9a74-7614a49e4edd'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:21.673131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-49437ecc-797c-4c68-9a74-7614a49e4edd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f971461bcef54bb5a7de9216db3925ce', 'request_id': 'req-49437ecc-797c-4c68-9a74-7614a49e4edd'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:21.682108 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-49437ecc-797c-4c68-9a74-7614a49e4edd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f971461bcef54bb5a7de9216db3925ce', 'request_id': 'req-49437ecc-797c-4c68-9a74-7614a49e4edd'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:21.707614 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-49437ecc-797c-4c68-9a74-7614a49e4edd admin admin] The token cache is being invalidate because project f971461bcef54bb5a7de9216db3925ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:26:21.708211 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-49437ecc-797c-4c68-9a74-7614a49e4edd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:21.709035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-49437ecc-797c-4c68-9a74-7614a49e4edd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:21.711750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 954/1915] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:26:21 2022] DELETE /identity/v3/projects/f971461bcef54bb5a7de9216db3925ce => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:21.718418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff8535-6222-44e0-a07a-4f5a1a2cceb3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:21.718915 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff8535-6222-44e0-a07a-4f5a1a2cceb3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:21.719221 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff8535-6222-44e0-a07a-4f5a1a2cceb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:21.773129 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-7dff8535-6222-44e0-a07a-4f5a1a2cceb3 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:21.775022 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7dff8535-6222-44e0-a07a-4f5a1a2cceb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:21.951423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 962/1916] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:21.953491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a377b-280d-425f-95ba-c6674297fee9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:21.953849 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a377b-280d-425f-95ba-c6674297fee9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:21.954118 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a377b-280d-425f-95ba-c6674297fee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:21.998277 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-479a377b-280d-425f-95ba-c6674297fee9 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:22.000234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-479a377b-280d-425f-95ba-c6674297fee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:22.022158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 955/1917] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:22.025913 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff8535-6222-44e0-a07a-4f5a1a2cceb3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:22.040223 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff8535-6222-44e0-a07a-4f5a1a2cceb3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:22.053686 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d804cc6-d5ac-4600-9874-e770a63e643c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:22.054889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d804cc6-d5ac-4600-9874-e770a63e643c admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:22.055541 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d804cc6-d5ac-4600-9874-e770a63e643c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:22.055999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d804cc6-d5ac-4600-9874-e770a63e643c admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:22.057236 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d804cc6-d5ac-4600-9874-e770a63e643c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:26:22.057730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d804cc6-d5ac-4600-9874-e770a63e643c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:22.060867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d804cc6-d5ac-4600-9874-e770a63e643c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:22.099569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a377b-280d-425f-95ba-c6674297fee9 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:22.100087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 963/1918] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:26:22.120334 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a377b-280d-425f-95ba-c6674297fee9 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:22.141279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbd808-d140-44bb-83ad-ed280dbf7fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:22.148497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbd808-d140-44bb-83ad-ed280dbf7fce admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:22.148791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbd808-d140-44bb-83ad-ed280dbf7fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:22.149064 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbd808-d140-44bb-83ad-ed280dbf7fce admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:22.149748 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbd808-d140-44bb-83ad-ed280dbf7fce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:22.151735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbd808-d140-44bb-83ad-ed280dbf7fce admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:22.282241 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 956/1919] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:26:22 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:22.287263 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d804cc6-d5ac-4600-9874-e770a63e643c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:22.302061 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d804cc6-d5ac-4600-9874-e770a63e643c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:22.475399 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aec0b6-b751-41b8-9001-f6acf3ffb3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:22.476320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aec0b6-b751-41b8-9001-f6acf3ffb3bd admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:22.476753 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aec0b6-b751-41b8-9001-f6acf3ffb3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:22.477177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aec0b6-b751-41b8-9001-f6acf3ffb3bd admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:22.486273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aec0b6-b751-41b8-9001-f6acf3ffb3bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:22.492492 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aec0b6-b751-41b8-9001-f6acf3ffb3bd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:22.554968 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 964/1920] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:26:22 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:22.555626 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbd808-d140-44bb-83ad-ed280dbf7fce admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:22.569844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbd808-d140-44bb-83ad-ed280dbf7fce admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:22.583795 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe8f4e-47d4-44f4-8cc9-384cf420503b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:22.584433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe8f4e-47d4-44f4-8cc9-384cf420503b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:22.584709 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe8f4e-47d4-44f4-8cc9-384cf420503b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:22.584977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe8f4e-47d4-44f4-8cc9-384cf420503b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:22.585639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe8f4e-47d4-44f4-8cc9-384cf420503b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:22.587296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe8f4e-47d4-44f4-8cc9-384cf420503b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:22.621333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 957/1921] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:22.622606 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aec0b6-b751-41b8-9001-f6acf3ffb3bd admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:22.653979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aec0b6-b751-41b8-9001-f6acf3ffb3bd admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:22.665370 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffd094-f18b-4e96-93d3-0b69a7b74454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:22.665997 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd094-f18b-4e96-93d3-0b69a7b74454 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:22.666274 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd094-f18b-4e96-93d3-0b69a7b74454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:22.666542 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd094-f18b-4e96-93d3-0b69a7b74454 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:22.667330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffd094-f18b-4e96-93d3-0b69a7b74454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:22.669001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffd094-f18b-4e96-93d3-0b69a7b74454 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:22.690944 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:22.710125 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe8f4e-47d4-44f4-8cc9-384cf420503b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:22.710561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 965/1922] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:22.746042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe8f4e-47d4-44f4-8cc9-384cf420503b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:22.756241 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5987e38-f5eb-47a5-b434-d422ea07e4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:22.757127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5987e38-f5eb-47a5-b434-d422ea07e4d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:22.757565 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5987e38-f5eb-47a5-b434-d422ea07e4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:22.757977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5987e38-f5eb-47a5-b434-d422ea07e4d5 admin admin] PATH_INFO: `/v3/projects/8e9fdd7fd41d4ab69bcd51da90c2418b/users/9d0a97dc50d24a6e82b029fd9f50177e/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:22.795874 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5987e38-f5eb-47a5-b434-d422ea07e4d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e9fdd7fd41d4ab69bcd51da90c2418b, user_id=9d0a97dc50d24a6e82b029fd9f50177e, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:22.799033 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5987e38-f5eb-47a5-b434-d422ea07e4d5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:22.801848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:22.820155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 958/1923] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:22 2022] PUT /identity/v3/projects/8e9fdd7fd41d4ab69bcd51da90c2418b/users/9d0a97dc50d24a6e82b029fd9f50177e/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:22.832292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6100c4-460b-4833-9d05-4b0e92317b8c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:22.833250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6100c4-460b-4833-9d05-4b0e92317b8c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:22.833665 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6100c4-460b-4833-9d05-4b0e92317b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:22.891391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ba6100c4-460b-4833-9d05-4b0e92317b8c None None] MFA Rules not processed for user `9d0a97dc50d24a6e82b029fd9f50177e`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:22.892612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ba6100c4-460b-4833-9d05-4b0e92317b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:23.034578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 966/1924] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:23.051907 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5987e38-f5eb-47a5-b434-d422ea07e4d5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:23.054205 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a5987e38-f5eb-47a5-b434-d422ea07e4d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:23.189215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5987e38-f5eb-47a5-b434-d422ea07e4d5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:23.193819 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872bed7-ba3e-47a4-87a3-81bc7817fdae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:23.195476 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872bed7-ba3e-47a4-87a3-81bc7817fdae service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:23.195830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872bed7-ba3e-47a4-87a3-81bc7817fdae service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:23.196170 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872bed7-ba3e-47a4-87a3-81bc7817fdae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:23.219535 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872bed7-ba3e-47a4-87a3-81bc7817fdae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:23.230638 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872bed7-ba3e-47a4-87a3-81bc7817fdae service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:23.287000 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 959/1925] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:26:23.299786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6100c4-460b-4833-9d05-4b0e92317b8c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:23.453825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6100c4-460b-4833-9d05-4b0e92317b8c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:23.475860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e0c5a-6a2d-4ac4-a817-794a91da4834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:23.476545 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e0c5a-6a2d-4ac4-a817-794a91da4834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:23.476808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e0c5a-6a2d-4ac4-a817-794a91da4834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:23.477080 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e0c5a-6a2d-4ac4-a817-794a91da4834 admin admin] PATH_INFO: `/v3/users/9d0a97dc50d24a6e82b029fd9f50177e` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:23.479565 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e0c5a-6a2d-4ac4-a817-794a91da4834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d0a97dc50d24a6e82b029fd9f50177e)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:23.485025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e0c5a-6a2d-4ac4-a817-794a91da4834 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:23.523954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f630744-0bcf-43c1-bc0d-d10f7aeb6093 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:23.524240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f630744-0bcf-43c1-bc0d-d10f7aeb6093 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:23.524514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f630744-0bcf-43c1-bc0d-d10f7aeb6093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:23.636792 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b93e0c5a-6a2d-4ac4-a817-794a91da4834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d0a97dc50d24a6e82b029fd9f50177e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:23.644528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-6f630744-0bcf-43c1-bc0d-d10f7aeb6093 None None] MFA Rules not processed for user `d9f9821cd90e4d9096cc01aee5c0e569`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:23.646020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6f630744-0bcf-43c1-bc0d-d10f7aeb6093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:23.653546 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b93e0c5a-6a2d-4ac4-a817-794a91da4834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d0a97dc50d24a6e82b029fd9f50177e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:23.693269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b93e0c5a-6a2d-4ac4-a817-794a91da4834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d0a97dc50d24a6e82b029fd9f50177e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:23.712122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b93e0c5a-6a2d-4ac4-a817-794a91da4834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d0a97dc50d24a6e82b029fd9f50177e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:23.712972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b93e0c5a-6a2d-4ac4-a817-794a91da4834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d0a97dc50d24a6e82b029fd9f50177e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:23.731161 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 967/1926] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:26:23 2022] DELETE /identity/v3/users/9d0a97dc50d24a6e82b029fd9f50177e => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:23.738829 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6b7a5-fc94-4b98-a105-3033fef4409b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:23.738829 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6b7a5-fc94-4b98-a105-3033fef4409b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:23.738829 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6b7a5-fc94-4b98-a105-3033fef4409b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:23.757718 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:23.767627 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6aa6b7a5-fc94-4b98-a105-3033fef4409b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:23.769623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6aa6b7a5-fc94-4b98-a105-3033fef4409b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:23.868597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 960/1927] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:23.883866 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f630744-0bcf-43c1-bc0d-d10f7aeb6093 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:23.885307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6f630744-0bcf-43c1-bc0d-d10f7aeb6093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:23.908212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 968/1928] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:23.930869 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6b7a5-fc94-4b98-a105-3033fef4409b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:23.932629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6aa6b7a5-fc94-4b98-a105-3033fef4409b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:24.095722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6b7a5-fc94-4b98-a105-3033fef4409b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:24.105886 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79886d4-a8f9-4e22-9026-c4ca19f3b59a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:24.107564 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79886d4-a8f9-4e22-9026-c4ca19f3b59a service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:24.109059 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79886d4-a8f9-4e22-9026-c4ca19f3b59a service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:24.109663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79886d4-a8f9-4e22-9026-c4ca19f3b59a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:24.129526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f630744-0bcf-43c1-bc0d-d10f7aeb6093 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:24.134143 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79886d4-a8f9-4e22-9026-c4ca19f3b59a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:24.135521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79886d4-a8f9-4e22-9026-c4ca19f3b59a service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:24.167675 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 969/1929] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:26:24.232583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:24.350378 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f933d34-46a2-4b01-8ed5-d4584057f7df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:24.351235 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f933d34-46a2-4b01-8ed5-d4584057f7df service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:24.351521 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f933d34-46a2-4b01-8ed5-d4584057f7df service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:24.351779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f933d34-46a2-4b01-8ed5-d4584057f7df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:24.474689 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f933d34-46a2-4b01-8ed5-d4584057f7df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:24.477300 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f933d34-46a2-4b01-8ed5-d4584057f7df service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:24.503963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 961/1930] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:26:25.309755 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79886d4-a8f9-4e22-9026-c4ca19f3b59a service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:25.313175 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a79886d4-a8f9-4e22-9026-c4ca19f3b59a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:25.344204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79886d4-a8f9-4e22-9026-c4ca19f3b59a service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:25.350473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df551d35-37d2-4ef2-a8a2-def2323dbb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:25.351525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df551d35-37d2-4ef2-a8a2-def2323dbb53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:25.351946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df551d35-37d2-4ef2-a8a2-def2323dbb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:25.352430 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df551d35-37d2-4ef2-a8a2-def2323dbb53 admin admin] PATH_INFO: `/v3/projects/8e9fdd7fd41d4ab69bcd51da90c2418b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:25.353807 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df551d35-37d2-4ef2-a8a2-def2323dbb53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e9fdd7fd41d4ab69bcd51da90c2418b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:25.356398 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df551d35-37d2-4ef2-a8a2-def2323dbb53 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:25.425296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc6cb9-e2e0-4dee-a52a-c9df4c4d3063 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:25.426368 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc6cb9-e2e0-4dee-a52a-c9df4c4d3063 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:25.426368 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc6cb9-e2e0-4dee-a52a-c9df4c4d3063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:25.520501 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a1cc6cb9-e2e0-4dee-a52a-c9df4c4d3063 None None] MFA Rules not processed for user `e7e25970bb534ea39821f9c9640dcb66`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:25.523163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a1cc6cb9-e2e0-4dee-a52a-c9df4c4d3063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:25.530501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-df551d35-37d2-4ef2-a8a2-def2323dbb53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e9fdd7fd41d4ab69bcd51da90c2418b', 'request_id': 'req-df551d35-37d2-4ef2-a8a2-def2323dbb53'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:25.541060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-df551d35-37d2-4ef2-a8a2-def2323dbb53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e9fdd7fd41d4ab69bcd51da90c2418b', 'request_id': 'req-df551d35-37d2-4ef2-a8a2-def2323dbb53'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:25.554842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-df551d35-37d2-4ef2-a8a2-def2323dbb53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e9fdd7fd41d4ab69bcd51da90c2418b', 'request_id': 'req-df551d35-37d2-4ef2-a8a2-def2323dbb53'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:25.586442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-df551d35-37d2-4ef2-a8a2-def2323dbb53 admin admin] The token cache is being invalidate because project 8e9fdd7fd41d4ab69bcd51da90c2418b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:26:25.587041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-df551d35-37d2-4ef2-a8a2-def2323dbb53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:25.587943 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-df551d35-37d2-4ef2-a8a2-def2323dbb53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:26:25.594055 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 970/1931] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:26:25 2022] DELETE /identity/v3/projects/8e9fdd7fd41d4ab69bcd51da90c2418b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:25.596064 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b316e-b3d1-4be2-a03a-c4f14853c7ef None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:25.596498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b316e-b3d1-4be2-a03a-c4f14853c7ef None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:25.596939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b316e-b3d1-4be2-a03a-c4f14853c7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:25.628829 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fa3b316e-b3d1-4be2-a03a-c4f14853c7ef None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:25.630636 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fa3b316e-b3d1-4be2-a03a-c4f14853c7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:25.683438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:25.694378 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 962/1932] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:26:25.711497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc6cb9-e2e0-4dee-a52a-c9df4c4d3063 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:25.714302 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a1cc6cb9-e2e0-4dee-a52a-c9df4c4d3063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:25.746630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 971/1933] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:25.757614 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc520600-03f5-44b2-954f-4e1f2fbc8cba None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:25.758080 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc520600-03f5-44b2-954f-4e1f2fbc8cba None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:25.758454 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc520600-03f5-44b2-954f-4e1f2fbc8cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:25.791684 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fc520600-03f5-44b2-954f-4e1f2fbc8cba None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:25.793241 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fc520600-03f5-44b2-954f-4e1f2fbc8cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:25.808131 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 972/1934] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:25.809513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc520600-03f5-44b2-954f-4e1f2fbc8cba None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:25.822110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc520600-03f5-44b2-954f-4e1f2fbc8cba None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:25.827789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba7474-8e42-477b-b330-793d36d91bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:25.829326 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba7474-8e42-477b-b330-793d36d91bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:25.829702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba7474-8e42-477b-b330-793d36d91bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:25.830071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba7474-8e42-477b-b330-793d36d91bf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:25.830828 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba7474-8e42-477b-b330-793d36d91bf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:26:25.831290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba7474-8e42-477b-b330-793d36d91bf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:25.833149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba7474-8e42-477b-b330-793d36d91bf6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:25.855829 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 973/1935] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:26:25.862004 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba7474-8e42-477b-b330-793d36d91bf6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:25.874293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba7474-8e42-477b-b330-793d36d91bf6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:25.881777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b5077-6cbd-42ec-bead-e7537106e10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:25.882550 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b5077-6cbd-42ec-bead-e7537106e10f admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:25.882812 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b5077-6cbd-42ec-bead-e7537106e10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:25.883176 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b5077-6cbd-42ec-bead-e7537106e10f admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:25.883930 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b5077-6cbd-42ec-bead-e7537106e10f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:25.885863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b5077-6cbd-42ec-bead-e7537106e10f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:25.904725 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:25.949029 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 974/1936] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:26:25 2022] POST /identity/v3/projects => generated 369 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:25.953269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8323db-ac95-4aba-9b86-5c121a16e9ff None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:25.953570 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8323db-ac95-4aba-9b86-5c121a16e9ff None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:25.953825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8323db-ac95-4aba-9b86-5c121a16e9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:25.973936 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc6cb9-e2e0-4dee-a52a-c9df4c4d3063 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:26.004754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0a8323db-ac95-4aba-9b86-5c121a16e9ff None None] MFA Rules not processed for user `6fa96892d57d4d969e01d9014a50a8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:26.006985 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0a8323db-ac95-4aba-9b86-5c121a16e9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:26.158463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 975/1937] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:26.158463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8323db-ac95-4aba-9b86-5c121a16e9ff None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:26.167273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5689b2-bf1c-4f9e-90b5-eed7d5d75d9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:26.168115 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5689b2-bf1c-4f9e-90b5-eed7d5d75d9a service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:26.168223 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5689b2-bf1c-4f9e-90b5-eed7d5d75d9a service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:26.168544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5689b2-bf1c-4f9e-90b5-eed7d5d75d9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:26.178258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ba5689b2-bf1c-4f9e-90b5-eed7d5d75d9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:26.340514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5689b2-bf1c-4f9e-90b5-eed7d5d75d9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:26.368945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5689b2-bf1c-4f9e-90b5-eed7d5d75d9a service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:26.389968 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8323db-ac95-4aba-9b86-5c121a16e9ff None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:26.393675 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 963/1938] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:26:26.394883 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5689b2-bf1c-4f9e-90b5-eed7d5d75d9a service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:26.397342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ba5689b2-bf1c-4f9e-90b5-eed7d5d75d9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:26.402546 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e95d90-fa81-4dc3-9cfe-c767cb9cba9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:26.403268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e95d90-fa81-4dc3-9cfe-c767cb9cba9b admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:26.403569 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e95d90-fa81-4dc3-9cfe-c767cb9cba9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:26.403854 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e95d90-fa81-4dc3-9cfe-c767cb9cba9b admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:26.404476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e95d90-fa81-4dc3-9cfe-c767cb9cba9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:26.406595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e95d90-fa81-4dc3-9cfe-c767cb9cba9b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:26.451223 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 976/1939] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:26:26 2022] POST /identity/v3/users => generated 285 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:26:26.453227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e95d90-fa81-4dc3-9cfe-c767cb9cba9b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:26.477691 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e95d90-fa81-4dc3-9cfe-c767cb9cba9b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:26.488902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78824ab1-529c-4a2f-8c7a-e53228a9d128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:26.489811 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78824ab1-529c-4a2f-8c7a-e53228a9d128 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:26.490088 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78824ab1-529c-4a2f-8c7a-e53228a9d128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:26.490354 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78824ab1-529c-4a2f-8c7a-e53228a9d128 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:26.491032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78824ab1-529c-4a2f-8c7a-e53228a9d128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:26.492688 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78824ab1-529c-4a2f-8c7a-e53228a9d128 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:26.494269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:26.521075 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 977/1940] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:26.522412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78824ab1-529c-4a2f-8c7a-e53228a9d128 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:26.547199 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78824ab1-529c-4a2f-8c7a-e53228a9d128 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:26.563670 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855e30e-3216-4ee1-9567-fd4fa9aa9e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:26.564299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855e30e-3216-4ee1-9567-fd4fa9aa9e7a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:26.564586 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855e30e-3216-4ee1-9567-fd4fa9aa9e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:26.564859 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855e30e-3216-4ee1-9567-fd4fa9aa9e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:26.565504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855e30e-3216-4ee1-9567-fd4fa9aa9e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:26.567162 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855e30e-3216-4ee1-9567-fd4fa9aa9e7a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:26.575197 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5689b2-bf1c-4f9e-90b5-eed7d5d75d9a service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:26.588438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 978/1941] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:26:26.591616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b24ee8-79d8-41e9-b034-bbc8cbbe5f6d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c05a1b303c2418fb2941b8ef56f8972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:26.592509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b24ee8-79d8-41e9-b034-bbc8cbbe5f6d service glance] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:26.592934 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b24ee8-79d8-41e9-b034-bbc8cbbe5f6d service glance] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:26.593347 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b24ee8-79d8-41e9-b034-bbc8cbbe5f6d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:26.594888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855e30e-3216-4ee1-9567-fd4fa9aa9e7a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:26.596967 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b4b24ee8-79d8-41e9-b034-bbc8cbbe5f6d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:26.626266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855e30e-3216-4ee1-9567-fd4fa9aa9e7a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:26.638731 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:26.656211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178ffe5-5e71-43f3-bbc4-c246ed615e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:26.656942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178ffe5-5e71-43f3-bbc4-c246ed615e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:26.657222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178ffe5-5e71-43f3-bbc4-c246ed615e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:26.657503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178ffe5-5e71-43f3-bbc4-c246ed615e54 admin admin] PATH_INFO: `/v3/projects/496e48c5edb04fffa5ea2a28acbb2cc8/users/d666afaaaf3c44c287dd8510a3ec355b/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:26.710182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178ffe5-5e71-43f3-bbc4-c246ed615e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=496e48c5edb04fffa5ea2a28acbb2cc8, user_id=d666afaaaf3c44c287dd8510a3ec355b, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:26.712956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178ffe5-5e71-43f3-bbc4-c246ed615e54 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:26.750988 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 979/1942] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:26:26 2022] PUT /identity/v3/projects/496e48c5edb04fffa5ea2a28acbb2cc8/users/d666afaaaf3c44c287dd8510a3ec355b/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:26:26.761910 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a487b3c-5ecc-4eb7-8898-5f6a1f4204f6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:26.762193 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a487b3c-5ecc-4eb7-8898-5f6a1f4204f6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:26.762471 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a487b3c-5ecc-4eb7-8898-5f6a1f4204f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:26.804398 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b24ee8-79d8-41e9-b034-bbc8cbbe5f6d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:26.805932 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b24ee8-79d8-41e9-b034-bbc8cbbe5f6d service glance] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:26.833140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 964/1943] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:26:26.851905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4a487b3c-5ecc-4eb7-8898-5f6a1f4204f6 None None] MFA Rules not processed for user `d666afaaaf3c44c287dd8510a3ec355b`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:26.853474 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4a487b3c-5ecc-4eb7-8898-5f6a1f4204f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:27.021696 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 980/1944] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:27.030418 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b24ee8-79d8-41e9-b034-bbc8cbbe5f6d service glance] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:27.041714 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b24ee8-79d8-41e9-b034-bbc8cbbe5f6d service glance] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:27.063214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb0f68-e864-4864-a12e-0e265a7d675d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:27.063517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb0f68-e864-4864-a12e-0e265a7d675d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:27.063863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb0f68-e864-4864-a12e-0e265a7d675d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:27.104628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-60cb0f68-e864-4864-a12e-0e265a7d675d None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:27.105768 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-60cb0f68-e864-4864-a12e-0e265a7d675d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:27.173732 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc3353-d184-42d2-9459-2abe887610c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c05a1b303c2418fb2941b8ef56f8972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:27.174369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3353-d184-42d2-9459-2abe887610c9 service glance] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:27.174699 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3353-d184-42d2-9459-2abe887610c9 service glance] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:27.174917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3353-d184-42d2-9459-2abe887610c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:27.195581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc3353-d184-42d2-9459-2abe887610c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:27.197038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc3353-d184-42d2-9459-2abe887610c9 service glance] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:27.228184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 965/1945] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:26:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:26:27.236610 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 981/1946] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:26:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:27.312978 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc3353-d184-42d2-9459-2abe887610c9 service glance] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:27.314586 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7efc3353-d184-42d2-9459-2abe887610c9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:27.557547 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc3353-d184-42d2-9459-2abe887610c9 service glance] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:27.574591 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf3b74-4a24-4eb3-bf93-777745f49fd1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:27.575410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf3b74-4a24-4eb3-bf93-777745f49fd1 service swift] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:27.575956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf3b74-4a24-4eb3-bf93-777745f49fd1 service swift] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:27.576046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf3b74-4a24-4eb3-bf93-777745f49fd1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:27.599514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf3b74-4a24-4eb3-bf93-777745f49fd1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:27.604144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf3b74-4a24-4eb3-bf93-777745f49fd1 service swift] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:27.650269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 966/1947] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:26:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:26:28.248193 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e97c9-8089-4e2e-a623-65ec43a4d4d0 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:28.249207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e97c9-8089-4e2e-a623-65ec43a4d4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:28.249306 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e97c9-8089-4e2e-a623-65ec43a4d4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:28.312865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-675e97c9-8089-4e2e-a623-65ec43a4d4d0 None None] MFA Rules not processed for user `d9f9821cd90e4d9096cc01aee5c0e569`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:28.317503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-675e97c9-8089-4e2e-a623-65ec43a4d4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:28.548582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:28.561780 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 982/1948] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:29.634789 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afef640-be42-439f-b083-78bb02bfd6a8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:29.635309 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afef640-be42-439f-b083-78bb02bfd6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:29.635567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afef640-be42-439f-b083-78bb02bfd6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:29.692798 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4afef640-be42-439f-b083-78bb02bfd6a8 None None] MFA Rules not processed for user `d40a56908e934174b1f7b20e62e2cfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:29.703440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4afef640-be42-439f-b083-78bb02bfd6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:29.851614 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:29.880027 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 967/1949] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:39.859153 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:26:43.203946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b761a6-316e-4a5d-9250-f8d71f0d9d29 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:43.204553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b761a6-316e-4a5d-9250-f8d71f0d9d29 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:43.204553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b761a6-316e-4a5d-9250-f8d71f0d9d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:43.229693 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b0b761a6-316e-4a5d-9250-f8d71f0d9d29 None None] MFA Rules not processed for user `5e7267cf9bbf4bea81bc949221f31e18`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:43.231038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b0b761a6-316e-4a5d-9250-f8d71f0d9d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:43.301433 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 983/1950] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:51.063791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9d2b1-fda6-4f55-b073-49763df0b819 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:51.064329 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9d2b1-fda6-4f55-b073-49763df0b819 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:51.064706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9d2b1-fda6-4f55-b073-49763df0b819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:51.092289 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-bfd9d2b1-fda6-4f55-b073-49763df0b819 None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:51.094161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bfd9d2b1-fda6-4f55-b073-49763df0b819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:51.102866 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 968/1951] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:26:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:26:51.116379 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b761a6-316e-4a5d-9250-f8d71f0d9d29 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:26:51.131703 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b761a6-316e-4a5d-9250-f8d71f0d9d29 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:26:51.137062 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26a8ee-04c5-46cb-bec7-92cc001aec3b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:26:51.137994 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26a8ee-04c5-46cb-bec7-92cc001aec3b service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:51.138291 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26a8ee-04c5-46cb-bec7-92cc001aec3b service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:51.139180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26a8ee-04c5-46cb-bec7-92cc001aec3b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:51.148612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26a8ee-04c5-46cb-bec7-92cc001aec3b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:26:51.150214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26a8ee-04c5-46cb-bec7-92cc001aec3b service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:26:51.165455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 984/1952] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:26:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 14 02:26:53.429947 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a37ac-d54d-45fe-9462-745e9d2af83c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:26:53.431216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a37ac-d54d-45fe-9462-745e9d2af83c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:26:53.433170 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a37ac-d54d-45fe-9462-745e9d2af83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:26:53.480833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1f2a37ac-d54d-45fe-9462-745e9d2af83c None None] MFA Rules not processed for user `7d4748cbe63d490b9d8b6914a27ff364`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:26:53.482896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1f2a37ac-d54d-45fe-9462-745e9d2af83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:26:53.601099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 969/1953] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:26:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:27:01.155383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=31 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:27:03.590080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=28 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:27:05.322914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26a8ee-04c5-46cb-bec7-92cc001aec3b service swift] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:05.386498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26a8ee-04c5-46cb-bec7-92cc001aec3b service swift] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:05.390905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530d5dc-3470-4cf6-a5cd-d18bb58bdd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:05.391623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530d5dc-3470-4cf6-a5cd-d18bb58bdd88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:05.391900 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530d5dc-3470-4cf6-a5cd-d18bb58bdd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:05.392170 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530d5dc-3470-4cf6-a5cd-d18bb58bdd88 admin admin] PATH_INFO: `/v3/users/d666afaaaf3c44c287dd8510a3ec355b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:05.393259 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530d5dc-3470-4cf6-a5cd-d18bb58bdd88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d666afaaaf3c44c287dd8510a3ec355b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:05.395204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530d5dc-3470-4cf6-a5cd-d18bb58bdd88 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:05.440096 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f530d5dc-3470-4cf6-a5cd-d18bb58bdd88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd666afaaaf3c44c287dd8510a3ec355b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:05.446956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f530d5dc-3470-4cf6-a5cd-d18bb58bdd88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd666afaaaf3c44c287dd8510a3ec355b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:05.458730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f530d5dc-3470-4cf6-a5cd-d18bb58bdd88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd666afaaaf3c44c287dd8510a3ec355b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:05.465458 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f530d5dc-3470-4cf6-a5cd-d18bb58bdd88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd666afaaaf3c44c287dd8510a3ec355b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:05.466111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f530d5dc-3470-4cf6-a5cd-d18bb58bdd88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd666afaaaf3c44c287dd8510a3ec355b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:05.472655 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 985/1954] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:27:05 2022] DELETE /identity/v3/users/d666afaaaf3c44c287dd8510a3ec355b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:27:05.477163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec080d-0dba-44ed-ae28-414fb4b8eb77 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:05.477644 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec080d-0dba-44ed-ae28-414fb4b8eb77 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:05.477745 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec080d-0dba-44ed-ae28-414fb4b8eb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:05.502541 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a1ec080d-0dba-44ed-ae28-414fb4b8eb77 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:27:05.503770 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a1ec080d-0dba-44ed-ae28-414fb4b8eb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:05.568141 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 970/1955] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:27:05.577955 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530d5dc-3470-4cf6-a5cd-d18bb58bdd88 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:05.579423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f530d5dc-3470-4cf6-a5cd-d18bb58bdd88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:05.638536 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530d5dc-3470-4cf6-a5cd-d18bb58bdd88 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:05.642269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202aa070-bac6-4047-a143-e51eb1fe2a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:05.642882 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202aa070-bac6-4047-a143-e51eb1fe2a1d service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:05.643196 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202aa070-bac6-4047-a143-e51eb1fe2a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:05.643472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202aa070-bac6-4047-a143-e51eb1fe2a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:05.653497 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202aa070-bac6-4047-a143-e51eb1fe2a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:05.654986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202aa070-bac6-4047-a143-e51eb1fe2a1d service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:05.665663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 986/1956] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:27:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:27:05.987018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec080d-0dba-44ed-ae28-414fb4b8eb77 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:05.989290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a1ec080d-0dba-44ed-ae28-414fb4b8eb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:06.004204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec080d-0dba-44ed-ae28-414fb4b8eb77 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:06.010841 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a2b12-7153-44f4-bb49-a35f25d963ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:06.011551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a2b12-7153-44f4-bb49-a35f25d963ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:06.011830 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a2b12-7153-44f4-bb49-a35f25d963ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:06.012616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a2b12-7153-44f4-bb49-a35f25d963ad admin admin] PATH_INFO: `/v3/projects/496e48c5edb04fffa5ea2a28acbb2cc8` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:06.014001 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a2b12-7153-44f4-bb49-a35f25d963ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=496e48c5edb04fffa5ea2a28acbb2cc8)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:06.016114 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a2b12-7153-44f4-bb49-a35f25d963ad admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:06.099247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c35a2b12-7153-44f4-bb49-a35f25d963ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '496e48c5edb04fffa5ea2a28acbb2cc8', 'request_id': 'req-c35a2b12-7153-44f4-bb49-a35f25d963ad'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:06.106448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c35a2b12-7153-44f4-bb49-a35f25d963ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '496e48c5edb04fffa5ea2a28acbb2cc8', 'request_id': 'req-c35a2b12-7153-44f4-bb49-a35f25d963ad'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:06.116682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c35a2b12-7153-44f4-bb49-a35f25d963ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '496e48c5edb04fffa5ea2a28acbb2cc8', 'request_id': 'req-c35a2b12-7153-44f4-bb49-a35f25d963ad'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:06.132465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c35a2b12-7153-44f4-bb49-a35f25d963ad admin admin] The token cache is being invalidate because project 496e48c5edb04fffa5ea2a28acbb2cc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:27:06.133637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c35a2b12-7153-44f4-bb49-a35f25d963ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:06.134307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c35a2b12-7153-44f4-bb49-a35f25d963ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:06.136677 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 971/1957] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:27:05 2022] DELETE /identity/v3/projects/496e48c5edb04fffa5ea2a28acbb2cc8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:27:06.142086 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e8474-b00c-41d7-bdb4-b01de35ae034 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:06.142376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e8474-b00c-41d7-bdb4-b01de35ae034 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:06.142656 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e8474-b00c-41d7-bdb4-b01de35ae034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:06.163477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d35e8474-b00c-41d7-bdb4-b01de35ae034 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:27:06.165016 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d35e8474-b00c-41d7-bdb4-b01de35ae034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:06.185054 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:27:06.231769 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 987/1958] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:27:06.239759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af9dea-86cc-4aba-b526-c50165d11ed6 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:06.240142 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af9dea-86cc-4aba-b526-c50165d11ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:06.240479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af9dea-86cc-4aba-b526-c50165d11ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:06.261439 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-71af9dea-86cc-4aba-b526-c50165d11ed6 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:27:06.263355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-71af9dea-86cc-4aba-b526-c50165d11ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:06.271509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 972/1959] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:27:06.275700 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e8474-b00c-41d7-bdb4-b01de35ae034 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:06.285852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e8474-b00c-41d7-bdb4-b01de35ae034 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:06.290706 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb42d0d-8d8e-4171-aead-19a7d8cfd45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:06.291352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb42d0d-8d8e-4171-aead-19a7d8cfd45b admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:06.291631 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb42d0d-8d8e-4171-aead-19a7d8cfd45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:06.291903 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb42d0d-8d8e-4171-aead-19a7d8cfd45b admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:06.292553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb42d0d-8d8e-4171-aead-19a7d8cfd45b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:27:06.292857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb42d0d-8d8e-4171-aead-19a7d8cfd45b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:06.294980 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb42d0d-8d8e-4171-aead-19a7d8cfd45b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:06.306926 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 988/1960] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:27:06.311340 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af9dea-86cc-4aba-b526-c50165d11ed6 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:06.320396 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af9dea-86cc-4aba-b526-c50165d11ed6 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:06.325937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa067a3-d981-4e71-af66-d0ce74ec80d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:06.326554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa067a3-d981-4e71-af66-d0ce74ec80d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:06.326836 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa067a3-d981-4e71-af66-d0ce74ec80d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:06.327135 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa067a3-d981-4e71-af66-d0ce74ec80d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:06.327801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa067a3-d981-4e71-af66-d0ce74ec80d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:06.329704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa067a3-d981-4e71-af66-d0ce74ec80d6 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:06.359695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 973/1961] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:27:06 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:27:06.363987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb42d0d-8d8e-4171-aead-19a7d8cfd45b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:06.374867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb42d0d-8d8e-4171-aead-19a7d8cfd45b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:06.439307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde3498-93c4-4e1a-8a3d-466a316532c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:06.439923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde3498-93c4-4e1a-8a3d-466a316532c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:06.440199 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde3498-93c4-4e1a-8a3d-466a316532c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:06.440464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde3498-93c4-4e1a-8a3d-466a316532c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:06.441090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde3498-93c4-4e1a-8a3d-466a316532c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:06.443156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde3498-93c4-4e1a-8a3d-466a316532c4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:06.464807 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 989/1962] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:27:06 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:27:06.469722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa067a3-d981-4e71-af66-d0ce74ec80d6 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:06.482485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa067a3-d981-4e71-af66-d0ce74ec80d6 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:06.487091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044dfe79-d7bc-4bf2-8bc1-1eae091e1292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:06.487723 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044dfe79-d7bc-4bf2-8bc1-1eae091e1292 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:06.488005 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044dfe79-d7bc-4bf2-8bc1-1eae091e1292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:06.488306 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044dfe79-d7bc-4bf2-8bc1-1eae091e1292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:06.489241 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044dfe79-d7bc-4bf2-8bc1-1eae091e1292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:06.490885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044dfe79-d7bc-4bf2-8bc1-1eae091e1292 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:06.502099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 974/1963] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:27:06.506038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde3498-93c4-4e1a-8a3d-466a316532c4 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:06.516276 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde3498-93c4-4e1a-8a3d-466a316532c4 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:06.520828 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d31b0c-0b91-4470-9708-4c85e4f2380b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:06.521432 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d31b0c-0b91-4470-9708-4c85e4f2380b admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:06.521706 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d31b0c-0b91-4470-9708-4c85e4f2380b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:06.521977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d31b0c-0b91-4470-9708-4c85e4f2380b admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:06.522614 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d31b0c-0b91-4470-9708-4c85e4f2380b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:06.524439 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d31b0c-0b91-4470-9708-4c85e4f2380b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:06.535940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 990/1964] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:27:06.540184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044dfe79-d7bc-4bf2-8bc1-1eae091e1292 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:06.550359 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044dfe79-d7bc-4bf2-8bc1-1eae091e1292 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:06.555129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395e60d-39d7-4ade-9b19-d88ab287ea04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:06.555866 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395e60d-39d7-4ade-9b19-d88ab287ea04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:06.556223 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395e60d-39d7-4ade-9b19-d88ab287ea04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:06.556503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395e60d-39d7-4ade-9b19-d88ab287ea04 admin admin] PATH_INFO: `/v3/projects/8c14506562f8441584565e93910d4549/users/a363b386c2d946d58c83c05bdf5b3c4c/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:06.571835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395e60d-39d7-4ade-9b19-d88ab287ea04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c14506562f8441584565e93910d4549, user_id=a363b386c2d946d58c83c05bdf5b3c4c, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:06.574729 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395e60d-39d7-4ade-9b19-d88ab287ea04 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:06.598854 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 975/1965] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:27:06 2022] PUT /identity/v3/projects/8c14506562f8441584565e93910d4549/users/a363b386c2d946d58c83c05bdf5b3c4c/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:27:06.600765 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465fe5d-3218-495e-b111-92cab004817d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:06.601060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465fe5d-3218-495e-b111-92cab004817d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:06.601334 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465fe5d-3218-495e-b111-92cab004817d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:06.631564 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3465fe5d-3218-495e-b111-92cab004817d None None] MFA Rules not processed for user `a363b386c2d946d58c83c05bdf5b3c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:27:06.632810 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3465fe5d-3218-495e-b111-92cab004817d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:06.701084 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 991/1966] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:27:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:27:06.712287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395e60d-39d7-4ade-9b19-d88ab287ea04 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:06.714057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8395e60d-39d7-4ade-9b19-d88ab287ea04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:06.752163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:27:06.804532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395e60d-39d7-4ade-9b19-d88ab287ea04 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:06.808117 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd74c5-1d43-4d31-b7e0-53f931e5bf2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:06.808854 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd74c5-1d43-4d31-b7e0-53f931e5bf2f service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:06.809513 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd74c5-1d43-4d31-b7e0-53f931e5bf2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:06.809513 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd74c5-1d43-4d31-b7e0-53f931e5bf2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:06.820503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd74c5-1d43-4d31-b7e0-53f931e5bf2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:06.822052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd74c5-1d43-4d31-b7e0-53f931e5bf2f service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:06.833085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 976/1967] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:27:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:27:06.862781 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465fe5d-3218-495e-b111-92cab004817d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:06.892862 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:27:06.942408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465fe5d-3218-495e-b111-92cab004817d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:06.946884 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bef38f-41ae-45e4-b651-1e60f611b88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:06.947559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bef38f-41ae-45e4-b651-1e60f611b88e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:06.947948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bef38f-41ae-45e4-b651-1e60f611b88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:06.948224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bef38f-41ae-45e4-b651-1e60f611b88e admin admin] PATH_INFO: `/v3/users/a363b386c2d946d58c83c05bdf5b3c4c` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:06.949343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bef38f-41ae-45e4-b651-1e60f611b88e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a363b386c2d946d58c83c05bdf5b3c4c)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:06.951965 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bef38f-41ae-45e4-b651-1e60f611b88e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:07.030559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-30bef38f-41ae-45e4-b651-1e60f611b88e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a363b386c2d946d58c83c05bdf5b3c4c'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:07.040385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-30bef38f-41ae-45e4-b651-1e60f611b88e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a363b386c2d946d58c83c05bdf5b3c4c'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:07.069719 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-30bef38f-41ae-45e4-b651-1e60f611b88e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a363b386c2d946d58c83c05bdf5b3c4c'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:07.079764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-30bef38f-41ae-45e4-b651-1e60f611b88e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a363b386c2d946d58c83c05bdf5b3c4c'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:07.080506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-30bef38f-41ae-45e4-b651-1e60f611b88e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a363b386c2d946d58c83c05bdf5b3c4c'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:07.088187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 992/1968] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:27:06 2022] DELETE /identity/v3/users/a363b386c2d946d58c83c05bdf5b3c4c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:27:07.092529 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59561167-9f5d-4326-b52d-7fbb6692f75a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:07.092813 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59561167-9f5d-4326-b52d-7fbb6692f75a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:07.093084 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59561167-9f5d-4326-b52d-7fbb6692f75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:07.118948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-59561167-9f5d-4326-b52d-7fbb6692f75a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:27:07.120265 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-59561167-9f5d-4326-b52d-7fbb6692f75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:07.192156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 977/1969] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:27:07.199252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bef38f-41ae-45e4-b651-1e60f611b88e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:07.200816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-30bef38f-41ae-45e4-b651-1e60f611b88e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:07.270010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bef38f-41ae-45e4-b651-1e60f611b88e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:07.273912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed21384-3ede-4faf-a32b-3dacd9d58c37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:07.274931 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed21384-3ede-4faf-a32b-3dacd9d58c37 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:07.275129 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed21384-3ede-4faf-a32b-3dacd9d58c37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:07.275403 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed21384-3ede-4faf-a32b-3dacd9d58c37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:07.286523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed21384-3ede-4faf-a32b-3dacd9d58c37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:07.287985 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed21384-3ede-4faf-a32b-3dacd9d58c37 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:07.297779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 993/1970] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:27:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:27:07.595615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59561167-9f5d-4326-b52d-7fbb6692f75a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:07.597093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-59561167-9f5d-4326-b52d-7fbb6692f75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:07.610375 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59561167-9f5d-4326-b52d-7fbb6692f75a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:07.616517 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:07.617373 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:07.617824 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:07.618325 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] PATH_INFO: `/v3/projects/8c14506562f8441584565e93910d4549` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:07.619532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c14506562f8441584565e93910d4549)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:07.622177 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:07.671867 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c14506562f8441584565e93910d4549', 'request_id': 'req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:07.678107 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c14506562f8441584565e93910d4549', 'request_id': 'req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:07.685047 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c14506562f8441584565e93910d4549', 'request_id': 'req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:07.698479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] The token cache is being invalidate because project 8c14506562f8441584565e93910d4549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:27:07.698801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:07.700401 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:27:07.702085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 978/1971] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:27:07 2022] DELETE /identity/v3/projects/8c14506562f8441584565e93910d4549 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:27:17.293565 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:27:17.695317 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:27:23.711327 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85735682-aea3-4f4c-a02c-b18f338c34ee None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:23.711817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85735682-aea3-4f4c-a02c-b18f338c34ee None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:23.711983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85735682-aea3-4f4c-a02c-b18f338c34ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:23.765361 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-85735682-aea3-4f4c-a02c-b18f338c34ee None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:27:23.767256 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-85735682-aea3-4f4c-a02c-b18f338c34ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:23.890071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 994/1972] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:27:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:27:23.900791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:23.903009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:24.018242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d785e29-fb98-4ccf-9af3-7a0b8d7205b6 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:24.024884 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84ccd8-9b0d-4d96-b91d-fadd85b09f91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:24.025652 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84ccd8-9b0d-4d96-b91d-fadd85b09f91 service swift] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:24.026027 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84ccd8-9b0d-4d96-b91d-fadd85b09f91 service swift] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:24.026551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84ccd8-9b0d-4d96-b91d-fadd85b09f91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:24.037186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84ccd8-9b0d-4d96-b91d-fadd85b09f91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:24.039174 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84ccd8-9b0d-4d96-b91d-fadd85b09f91 service swift] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:24.049892 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 979/1973] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:27:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:27:27.654813 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a4744-06cd-4d2a-9625-c77dff32b917 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:27.655460 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a4744-06cd-4d2a-9625-c77dff32b917 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:27.655561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a4744-06cd-4d2a-9625-c77dff32b917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:27.695086 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8a0a4744-06cd-4d2a-9625-c77dff32b917 None None] MFA Rules not processed for user `79365a5e4ae2431d91a4aa603ba58fee`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:27:27.695086 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8a0a4744-06cd-4d2a-9625-c77dff32b917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:27.805777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 995/1974] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:27:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:27:30.025192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84ccd8-9b0d-4d96-b91d-fadd85b09f91 service swift] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:30.028115 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4d84ccd8-9b0d-4d96-b91d-fadd85b09f91 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:30.151530 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84ccd8-9b0d-4d96-b91d-fadd85b09f91 service swift] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:30.157488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b850b-87b6-4d3b-94d7-02c806f88068 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e935ba63cdd4256874b99a2eb0a1a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:30.158415 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b850b-87b6-4d3b-94d7-02c806f88068 service placement] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:30.158917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b850b-87b6-4d3b-94d7-02c806f88068 service placement] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:30.159440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b850b-87b6-4d3b-94d7-02c806f88068 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:30.162136 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-243b850b-87b6-4d3b-94d7-02c806f88068 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:30.178369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b850b-87b6-4d3b-94d7-02c806f88068 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:30.180788 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b850b-87b6-4d3b-94d7-02c806f88068 service placement] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:30.216060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 980/1975] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:27:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:27:33.924456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91119e9-c65b-47a5-afe7-c8e13b57436f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:33.925353 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91119e9-c65b-47a5-afe7-c8e13b57436f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:33.925845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91119e9-c65b-47a5-afe7-c8e13b57436f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:33.952692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a91119e9-c65b-47a5-afe7-c8e13b57436f None None] MFA Rules not processed for user `ca65b084a0974a5ba648963f71047345`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:27:33.954222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a91119e9-c65b-47a5-afe7-c8e13b57436f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:34.122971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 996/1976] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:27:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:27:40.190663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=64 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:27:44.107336 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=75 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:27:45.536410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b444da3-9c9a-4c43-98c8-e2cfab68bf03 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:45.536941 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b444da3-9c9a-4c43-98c8-e2cfab68bf03 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:45.538016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b444da3-9c9a-4c43-98c8-e2cfab68bf03 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:45.540165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 981/1977] 10.222.0.18 () {54 vars in 921 bytes} [Fri Oct 14 02:27:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:27:45.546998 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160393c-6635-4f3c-afe5-0b085c1fee2f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:45.547504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160393c-6635-4f3c-afe5-0b085c1fee2f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:45.547682 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160393c-6635-4f3c-afe5-0b085c1fee2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:45.571504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9160393c-6635-4f3c-afe5-0b085c1fee2f None None] MFA Rules not processed for user `de13be55b4034477baf4c029c1a79757`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:27:45.573298 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9160393c-6635-4f3c-afe5-0b085c1fee2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:45.703915 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 997/1978] 10.222.0.18 () {58 vars in 1018 bytes} [Fri Oct 14 02:27:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:27:45.715548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b444da3-9c9a-4c43-98c8-e2cfab68bf03 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:45.717293 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6b444da3-9c9a-4c43-98c8-e2cfab68bf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:45.821053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b444da3-9c9a-4c43-98c8-e2cfab68bf03 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:45.825465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc0691-b41b-4441-b6b7-9c7b1fa72f22 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c05a1b303c2418fb2941b8ef56f8972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:45.826091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc0691-b41b-4441-b6b7-9c7b1fa72f22 service glance] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:45.826370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc0691-b41b-4441-b6b7-9c7b1fa72f22 service glance] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:45.826642 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc0691-b41b-4441-b6b7-9c7b1fa72f22 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:45.839448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc0691-b41b-4441-b6b7-9c7b1fa72f22 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:45.840892 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc0691-b41b-4441-b6b7-9c7b1fa72f22 service glance] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:45.851782 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 982/1979] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:27:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:27:45.923291 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf0a7b-1390-4640-8b1b-d7e3cde35a9c None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:45.923589 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf0a7b-1390-4640-8b1b-d7e3cde35a9c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:45.923929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf0a7b-1390-4640-8b1b-d7e3cde35a9c None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:45.925464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 998/1980] 10.222.0.18 () {54 vars in 921 bytes} [Fri Oct 14 02:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:27:45.930723 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0875e-bda6-4052-96fe-b9d3060f551d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:45.931220 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0875e-bda6-4052-96fe-b9d3060f551d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:45.931500 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0875e-bda6-4052-96fe-b9d3060f551d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:45.957388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-48b0875e-bda6-4052-96fe-b9d3060f551d None None] MFA Rules not processed for user `6ff451e43cea4bb0a5fd73491a05bf82`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:27:45.958578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-48b0875e-bda6-4052-96fe-b9d3060f551d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:46.009872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 983/1981] 10.222.0.18 () {58 vars in 1018 bytes} [Fri Oct 14 02:27:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:27:46.021769 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf0a7b-1390-4640-8b1b-d7e3cde35a9c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:46.023496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-70cf0a7b-1390-4640-8b1b-d7e3cde35a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:46.036630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf0a7b-1390-4640-8b1b-d7e3cde35a9c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:46.040310 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11a7df-f278-46d0-8265-4e133c46d471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:46.040917 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11a7df-f278-46d0-8265-4e133c46d471 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:46.041747 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11a7df-f278-46d0-8265-4e133c46d471 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:46.042334 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11a7df-f278-46d0-8265-4e133c46d471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:46.049424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11a7df-f278-46d0-8265-4e133c46d471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:46.050850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11a7df-f278-46d0-8265-4e133c46d471 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:46.059607 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 999/1982] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:27:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:27:46.146746 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763559aa-4913-4372-9783-bfe119a7a258 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:46.147059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763559aa-4913-4372-9783-bfe119a7a258 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:46.147352 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763559aa-4913-4372-9783-bfe119a7a258 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:46.148698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 984/1983] 10.222.0.18 () {54 vars in 921 bytes} [Fri Oct 14 02:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:27:46.154860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10f98b-13fa-4728-84e5-a6291f13b167 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:46.155200 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10f98b-13fa-4728-84e5-a6291f13b167 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:46.155525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10f98b-13fa-4728-84e5-a6291f13b167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:46.176439 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ad10f98b-13fa-4728-84e5-a6291f13b167 None None] MFA Rules not processed for user `de13be55b4034477baf4c029c1a79757`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:27:46.177580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ad10f98b-13fa-4728-84e5-a6291f13b167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:46.187528 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1000/1984] 10.222.0.18 () {58 vars in 1018 bytes} [Fri Oct 14 02:27:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:27:46.199081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763559aa-4913-4372-9783-bfe119a7a258 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:46.200766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-763559aa-4913-4372-9783-bfe119a7a258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:46.214717 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763559aa-4913-4372-9783-bfe119a7a258 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:46.218550 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea0dca-6d67-400e-93e7-58f052cba53c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:46.219230 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea0dca-6d67-400e-93e7-58f052cba53c service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:46.219512 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea0dca-6d67-400e-93e7-58f052cba53c service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:46.219789 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea0dca-6d67-400e-93e7-58f052cba53c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:46.230843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea0dca-6d67-400e-93e7-58f052cba53c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:46.232299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea0dca-6d67-400e-93e7-58f052cba53c service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:46.243348 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 985/1985] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:27:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:27:49.878243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10f98b-13fa-4728-84e5-a6291f13b167 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:49.887732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10f98b-13fa-4728-84e5-a6291f13b167 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:49.891707 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110d47b-b0f8-4e8d-8cf5-6194011d4d20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:49.892318 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110d47b-b0f8-4e8d-8cf5-6194011d4d20 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:49.892812 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110d47b-b0f8-4e8d-8cf5-6194011d4d20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:49.892918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110d47b-b0f8-4e8d-8cf5-6194011d4d20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:49.894619 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7110d47b-b0f8-4e8d-8cf5-6194011d4d20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:49.958135 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110d47b-b0f8-4e8d-8cf5-6194011d4d20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:49.959603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110d47b-b0f8-4e8d-8cf5-6194011d4d20 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:49.969541 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1001/1986] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:27:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:27:51.475927 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea0dca-6d67-400e-93e7-58f052cba53c service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:51.496279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea0dca-6d67-400e-93e7-58f052cba53c service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:51.501427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2007c-bc6f-4024-bbd3-0e1bd0f973b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:51.502382 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2007c-bc6f-4024-bbd3-0e1bd0f973b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:51.502761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2007c-bc6f-4024-bbd3-0e1bd0f973b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:51.503122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2007c-bc6f-4024-bbd3-0e1bd0f973b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:51.505377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-47f2007c-bc6f-4024-bbd3-0e1bd0f973b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:51.579450 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2007c-bc6f-4024-bbd3-0e1bd0f973b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:51.580927 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2007c-bc6f-4024-bbd3-0e1bd0f973b8 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:51.593126 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 986/1987] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:27:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:27:56.001017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d8f84-28ce-4f11-8789-0fd6a4438e43 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:56.001531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d8f84-28ce-4f11-8789-0fd6a4438e43 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:56.001843 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d8f84-28ce-4f11-8789-0fd6a4438e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:56.030149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bf6d8f84-28ce-4f11-8789-0fd6a4438e43 None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:27:56.031464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bf6d8f84-28ce-4f11-8789-0fd6a4438e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:27:56.040128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1002/1988] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:27:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:27:56.058249 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2007c-bc6f-4024-bbd3-0e1bd0f973b8 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:27:56.071368 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2007c-bc6f-4024-bbd3-0e1bd0f973b8 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:27:56.076533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea499ff3-11fa-40a9-9723-abeeb6e442a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:27:56.077224 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea499ff3-11fa-40a9-9723-abeeb6e442a5 service swift] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:27:56.078398 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea499ff3-11fa-40a9-9723-abeeb6e442a5 service swift] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:27:56.078807 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea499ff3-11fa-40a9-9723-abeeb6e442a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:27:56.088554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea499ff3-11fa-40a9-9723-abeeb6e442a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:27:56.090254 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea499ff3-11fa-40a9-9723-abeeb6e442a5 service swift] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:27:56.099466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 987/1989] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:27:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:28:06.023292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=73 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:28:06.095885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:28:30.763303 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740f09d-1a2b-4b59-b1f2-86b1e40dccb4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:28:30.763801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740f09d-1a2b-4b59-b1f2-86b1e40dccb4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:28:30.763888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740f09d-1a2b-4b59-b1f2-86b1e40dccb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:28:30.782647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a740f09d-1a2b-4b59-b1f2-86b1e40dccb4 None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:28:30.783870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a740f09d-1a2b-4b59-b1f2-86b1e40dccb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:28:30.791431 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1003/1990] 10.222.0.18 () {58 vars in 967 bytes} [Fri Oct 14 02:28:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:28:30.813680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea499ff3-11fa-40a9-9723-abeeb6e442a5 service swift] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:28:30.824184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea499ff3-11fa-40a9-9723-abeeb6e442a5 service swift] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:28:30.829134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef27450-fd2f-4d82-9b88-398d04c12fc5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:28:30.829728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef27450-fd2f-4d82-9b88-398d04c12fc5 service swift] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:28:30.830005 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef27450-fd2f-4d82-9b88-398d04c12fc5 service swift] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:28:30.830274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef27450-fd2f-4d82-9b88-398d04c12fc5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:28:30.838208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef27450-fd2f-4d82-9b88-398d04c12fc5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:28:30.839810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef27450-fd2f-4d82-9b88-398d04c12fc5 service swift] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:28:30.847577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 988/1991] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:28:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:28:40.777883 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:28:40.844454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=6 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:29:00.658678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740f09d-1a2b-4b59-b1f2-86b1e40dccb4 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:00.669117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a740f09d-1a2b-4b59-b1f2-86b1e40dccb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:00.779345 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740f09d-1a2b-4b59-b1f2-86b1e40dccb4 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:00.784228 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088ea64-83c1-4e2f-b559-732afc23a1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:00.784866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088ea64-83c1-4e2f-b559-732afc23a1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:00.785336 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088ea64-83c1-4e2f-b559-732afc23a1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:00.785756 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088ea64-83c1-4e2f-b559-732afc23a1e4 admin admin] PATH_INFO: `/v3/users/ca65b084a0974a5ba648963f71047345` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:00.787027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088ea64-83c1-4e2f-b559-732afc23a1e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca65b084a0974a5ba648963f71047345)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:00.789145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088ea64-83c1-4e2f-b559-732afc23a1e4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:00.851557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6088ea64-83c1-4e2f-b559-732afc23a1e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca65b084a0974a5ba648963f71047345'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:00.860098 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6088ea64-83c1-4e2f-b559-732afc23a1e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca65b084a0974a5ba648963f71047345'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:00.875010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6088ea64-83c1-4e2f-b559-732afc23a1e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca65b084a0974a5ba648963f71047345'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:00.883033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6088ea64-83c1-4e2f-b559-732afc23a1e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca65b084a0974a5ba648963f71047345'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:00.883786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6088ea64-83c1-4e2f-b559-732afc23a1e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca65b084a0974a5ba648963f71047345'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:00.891286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1004/1992] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:29:00 2022] DELETE /identity/v3/users/ca65b084a0974a5ba648963f71047345 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:29:01.065984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef27450-fd2f-4d82-9b88-398d04c12fc5 service swift] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:01.080809 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef27450-fd2f-4d82-9b88-398d04c12fc5 service swift] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:01.162220 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f709c04-d4b8-462c-bb7c-f679c3a25441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:01.162905 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f709c04-d4b8-462c-bb7c-f679c3a25441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:01.163650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f709c04-d4b8-462c-bb7c-f679c3a25441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:01.163650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f709c04-d4b8-462c-bb7c-f679c3a25441 admin admin] PATH_INFO: `/v3/projects/d4b1c4a008a84d9587b98c155ac0ace1` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:01.164898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f709c04-d4b8-462c-bb7c-f679c3a25441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4b1c4a008a84d9587b98c155ac0ace1)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:01.167377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f709c04-d4b8-462c-bb7c-f679c3a25441 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:01.235547 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8f709c04-d4b8-462c-bb7c-f679c3a25441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4b1c4a008a84d9587b98c155ac0ace1', 'request_id': 'req-8f709c04-d4b8-462c-bb7c-f679c3a25441'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:01.245010 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8f709c04-d4b8-462c-bb7c-f679c3a25441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4b1c4a008a84d9587b98c155ac0ace1', 'request_id': 'req-8f709c04-d4b8-462c-bb7c-f679c3a25441'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:01.251755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8f709c04-d4b8-462c-bb7c-f679c3a25441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4b1c4a008a84d9587b98c155ac0ace1', 'request_id': 'req-8f709c04-d4b8-462c-bb7c-f679c3a25441'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:01.265256 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8f709c04-d4b8-462c-bb7c-f679c3a25441 admin admin] The token cache is being invalidate because project d4b1c4a008a84d9587b98c155ac0ace1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:29:01.265635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8f709c04-d4b8-462c-bb7c-f679c3a25441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:01.266328 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8f709c04-d4b8-462c-bb7c-f679c3a25441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:01.267934 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 989/1993] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:29:01 2022] DELETE /identity/v3/projects/d4b1c4a008a84d9587b98c155ac0ace1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:29:01.272801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a2ca7-ab30-4c46-afbb-4c910c57dbfc None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:01.273098 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a2ca7-ab30-4c46-afbb-4c910c57dbfc None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:01.273369 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a2ca7-ab30-4c46-afbb-4c910c57dbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:01.297041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a16a2ca7-ab30-4c46-afbb-4c910c57dbfc None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:29:01.298461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a16a2ca7-ab30-4c46-afbb-4c910c57dbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:01.373123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1005/1994] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:29:01.380792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c93b-7b1e-46e0-b79e-6d5568f5d5aa None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:01.381165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c93b-7b1e-46e0-b79e-6d5568f5d5aa None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:01.381647 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c93b-7b1e-46e0-b79e-6d5568f5d5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:01.409522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-6486c93b-7b1e-46e0-b79e-6d5568f5d5aa None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:29:01.410920 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6486c93b-7b1e-46e0-b79e-6d5568f5d5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:01.419739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 990/1995] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:29:01.424039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a2ca7-ab30-4c46-afbb-4c910c57dbfc None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:01.436148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a2ca7-ab30-4c46-afbb-4c910c57dbfc None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:01.440898 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19808620-dc5b-4361-9a88-f4437984f36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:01.441705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19808620-dc5b-4361-9a88-f4437984f36f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:01.441988 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19808620-dc5b-4361-9a88-f4437984f36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:01.443377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19808620-dc5b-4361-9a88-f4437984f36f admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:01.444054 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19808620-dc5b-4361-9a88-f4437984f36f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:29:01.444416 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19808620-dc5b-4361-9a88-f4437984f36f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:01.446274 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19808620-dc5b-4361-9a88-f4437984f36f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:01.460592 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1006/1996] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:29:01.465910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486c93b-7b1e-46e0-b79e-6d5568f5d5aa None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:01.479946 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486c93b-7b1e-46e0-b79e-6d5568f5d5aa None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:01.485086 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5facb3-1ca8-4ccf-ac41-8d4008e1112b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:01.485712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5facb3-1ca8-4ccf-ac41-8d4008e1112b admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:01.485995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5facb3-1ca8-4ccf-ac41-8d4008e1112b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:01.486272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5facb3-1ca8-4ccf-ac41-8d4008e1112b admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:01.487247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5facb3-1ca8-4ccf-ac41-8d4008e1112b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:01.489272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5facb3-1ca8-4ccf-ac41-8d4008e1112b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:01.517437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 991/1997] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:29:01 2022] POST /identity/v3/projects => generated 397 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:29:01.521725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19808620-dc5b-4361-9a88-f4437984f36f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:01.533202 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19808620-dc5b-4361-9a88-f4437984f36f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:01.536837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:29:01.594385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3250f-9a81-4b37-97df-f843186f1790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:01.594940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3250f-9a81-4b37-97df-f843186f1790 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:01.595251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3250f-9a81-4b37-97df-f843186f1790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:01.595517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3250f-9a81-4b37-97df-f843186f1790 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:01.596145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3250f-9a81-4b37-97df-f843186f1790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:01.598251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3250f-9a81-4b37-97df-f843186f1790 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:01.621507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1007/1998] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:29:01 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:29:01.625329 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5facb3-1ca8-4ccf-ac41-8d4008e1112b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:01.637694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5facb3-1ca8-4ccf-ac41-8d4008e1112b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:01.642290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff03f0-38ce-43dd-ac05-ec00c89a99e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:01.642900 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff03f0-38ce-43dd-ac05-ec00c89a99e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:01.643200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff03f0-38ce-43dd-ac05-ec00c89a99e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:01.643472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff03f0-38ce-43dd-ac05-ec00c89a99e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:01.644106 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff03f0-38ce-43dd-ac05-ec00c89a99e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:01.645716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff03f0-38ce-43dd-ac05-ec00c89a99e5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:01.656122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 992/1999] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:29:01.660110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3250f-9a81-4b37-97df-f843186f1790 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:01.670145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3250f-9a81-4b37-97df-f843186f1790 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:01.674778 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fb1c7-a2a8-452f-854b-2b2812592aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:01.675409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fb1c7-a2a8-452f-854b-2b2812592aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:01.675682 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fb1c7-a2a8-452f-854b-2b2812592aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:01.675948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fb1c7-a2a8-452f-854b-2b2812592aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:01.676579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fb1c7-a2a8-452f-854b-2b2812592aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:01.678164 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fb1c7-a2a8-452f-854b-2b2812592aa3 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:01.688705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1008/2000] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:29:01.692641 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff03f0-38ce-43dd-ac05-ec00c89a99e5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:01.702874 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff03f0-38ce-43dd-ac05-ec00c89a99e5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:01.707404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb9ec4-16bc-459d-8780-353ebfe05bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:01.708045 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb9ec4-16bc-459d-8780-353ebfe05bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:01.708320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb9ec4-16bc-459d-8780-353ebfe05bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:01.708597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb9ec4-16bc-459d-8780-353ebfe05bf3 admin admin] PATH_INFO: `/v3/projects/62a43fae652c4b308475b7bc8286baed/users/497ba11a29124b9ca4c90bcfe02ea58c/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:01.723416 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb9ec4-16bc-459d-8780-353ebfe05bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a43fae652c4b308475b7bc8286baed, user_id=497ba11a29124b9ca4c90bcfe02ea58c, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:01.726057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb9ec4-16bc-459d-8780-353ebfe05bf3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:01.739794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 993/2001] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:29:01 2022] PUT /identity/v3/projects/62a43fae652c4b308475b7bc8286baed/users/497ba11a29124b9ca4c90bcfe02ea58c/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:29:01.750897 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f368e46-8fbc-48fe-b8d9-655e20fd83a6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:01.751266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f368e46-8fbc-48fe-b8d9-655e20fd83a6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:01.751543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f368e46-8fbc-48fe-b8d9-655e20fd83a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:01.780387 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4f368e46-8fbc-48fe-b8d9-655e20fd83a6 None None] MFA Rules not processed for user `497ba11a29124b9ca4c90bcfe02ea58c`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:29:01.781574 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4f368e46-8fbc-48fe-b8d9-655e20fd83a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:01.845511 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1009/2002] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:29:01 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:29:01.856655 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb9ec4-16bc-459d-8780-353ebfe05bf3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:01.859720 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-09bb9ec4-16bc-459d-8780-353ebfe05bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:01.893518 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:29:01.923466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb9ec4-16bc-459d-8780-353ebfe05bf3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:01.928175 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be59cf2-a8f1-463c-b7b5-8e33e0feeae3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:01.928796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be59cf2-a8f1-463c-b7b5-8e33e0feeae3 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:01.929077 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be59cf2-a8f1-463c-b7b5-8e33e0feeae3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:01.929351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be59cf2-a8f1-463c-b7b5-8e33e0feeae3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:01.938684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be59cf2-a8f1-463c-b7b5-8e33e0feeae3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:01.940145 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be59cf2-a8f1-463c-b7b5-8e33e0feeae3 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:01.949667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 994/2003] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:29:01 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:29:03.961215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5fb75-3c15-4251-b9fb-ef4e40e0bd2b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:03.961738 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5fb75-3c15-4251-b9fb-ef4e40e0bd2b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:03.961738 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5fb75-3c15-4251-b9fb-ef4e40e0bd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:03.981232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-46c5fb75-3c15-4251-b9fb-ef4e40e0bd2b None None] MFA Rules not processed for user `7d4748cbe63d490b9d8b6914a27ff364`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:29:03.982410 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-46c5fb75-3c15-4251-b9fb-ef4e40e0bd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:04.035607 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1010/2004] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:29:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:29:09.092437 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be59cf2-a8f1-463c-b7b5-8e33e0feeae3 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:09.167141 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be59cf2-a8f1-463c-b7b5-8e33e0feeae3 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:09.172331 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53584a-cd05-44d1-abaf-0b69c960b855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:09.173150 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53584a-cd05-44d1-abaf-0b69c960b855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:09.173527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53584a-cd05-44d1-abaf-0b69c960b855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:09.173903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53584a-cd05-44d1-abaf-0b69c960b855 admin admin] PATH_INFO: `/v3/users/497ba11a29124b9ca4c90bcfe02ea58c` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:09.175196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53584a-cd05-44d1-abaf-0b69c960b855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=497ba11a29124b9ca4c90bcfe02ea58c)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:09.177305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53584a-cd05-44d1-abaf-0b69c960b855 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:09.235954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2b53584a-cd05-44d1-abaf-0b69c960b855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '497ba11a29124b9ca4c90bcfe02ea58c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:09.243843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2b53584a-cd05-44d1-abaf-0b69c960b855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '497ba11a29124b9ca4c90bcfe02ea58c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:09.244593 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2b53584a-cd05-44d1-abaf-0b69c960b855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '497ba11a29124b9ca4c90bcfe02ea58c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:09.260071 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2b53584a-cd05-44d1-abaf-0b69c960b855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '497ba11a29124b9ca4c90bcfe02ea58c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:09.270244 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2b53584a-cd05-44d1-abaf-0b69c960b855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '497ba11a29124b9ca4c90bcfe02ea58c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:09.278322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 995/2005] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:29:09 2022] DELETE /identity/v3/users/497ba11a29124b9ca4c90bcfe02ea58c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:29:09.283262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cc802-da1c-41bc-a892-110cc3a072d3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:09.283678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cc802-da1c-41bc-a892-110cc3a072d3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:09.283678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cc802-da1c-41bc-a892-110cc3a072d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:09.317208 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-aa7cc802-da1c-41bc-a892-110cc3a072d3 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:29:09.318796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-aa7cc802-da1c-41bc-a892-110cc3a072d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:09.348729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:29:09.402873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1011/2006] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:29:09.412690 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53584a-cd05-44d1-abaf-0b69c960b855 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:09.414098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2b53584a-cd05-44d1-abaf-0b69c960b855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:09.468366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:29:09.472938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53584a-cd05-44d1-abaf-0b69c960b855 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:09.478217 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62a9aa-50ac-4380-a461-2acb815d7363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:09.479282 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62a9aa-50ac-4380-a461-2acb815d7363 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:09.480196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62a9aa-50ac-4380-a461-2acb815d7363 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:09.480538 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62a9aa-50ac-4380-a461-2acb815d7363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:09.490696 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62a9aa-50ac-4380-a461-2acb815d7363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:09.492164 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62a9aa-50ac-4380-a461-2acb815d7363 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:09.503025 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 996/2007] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:29:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:29:09.852066 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cc802-da1c-41bc-a892-110cc3a072d3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:09.854106 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-aa7cc802-da1c-41bc-a892-110cc3a072d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:09.866530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cc802-da1c-41bc-a892-110cc3a072d3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:09.871886 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:09.872616 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:09.872972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:09.873308 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] PATH_INFO: `/v3/projects/62a43fae652c4b308475b7bc8286baed` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:09.874485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a43fae652c4b308475b7bc8286baed)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:09.876501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:09.922215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a43fae652c4b308475b7bc8286baed', 'request_id': 'req-804d0da7-a128-4277-9aad-4bdfbf515a82'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:09.927681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a43fae652c4b308475b7bc8286baed', 'request_id': 'req-804d0da7-a128-4277-9aad-4bdfbf515a82'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:09.934811 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a43fae652c4b308475b7bc8286baed', 'request_id': 'req-804d0da7-a128-4277-9aad-4bdfbf515a82'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:09.947425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] The token cache is being invalidate because project 62a43fae652c4b308475b7bc8286baed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:29:09.947708 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:09.948322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:29:09.949822 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1012/2008] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:29:09 2022] DELETE /identity/v3/projects/62a43fae652c4b308475b7bc8286baed => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:29:19.498673 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:29:19.944244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:29:21.417152 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62a9aa-50ac-4380-a461-2acb815d7363 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:21.425731 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62a9aa-50ac-4380-a461-2acb815d7363 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:21.474397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdac5f-812f-4e84-aaae-40490b3ddc67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:21.475002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdac5f-812f-4e84-aaae-40490b3ddc67 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:21.475308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdac5f-812f-4e84-aaae-40490b3ddc67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:21.475577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdac5f-812f-4e84-aaae-40490b3ddc67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:21.477321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9acdac5f-812f-4e84-aaae-40490b3ddc67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:21.550341 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acdac5f-812f-4e84-aaae-40490b3ddc67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:21.551802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acdac5f-812f-4e84-aaae-40490b3ddc67 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:21.563054 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 997/2009] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:29:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:29:22.744827 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:22.746493 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:22.762176 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d0da7-a128-4277-9aad-4bdfbf515a82 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:22.766527 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4a646-93f3-4e3f-880f-df69b3b33da8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:22.767179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4a646-93f3-4e3f-880f-df69b3b33da8 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:22.767441 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4a646-93f3-4e3f-880f-df69b3b33da8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:22.767709 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4a646-93f3-4e3f-880f-df69b3b33da8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:22.769454 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-86f4a646-93f3-4e3f-880f-df69b3b33da8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:22.783617 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4a646-93f3-4e3f-880f-df69b3b33da8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:22.785066 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4a646-93f3-4e3f-880f-df69b3b33da8 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:22.798539 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1013/2010] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:29:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:29:29.251683 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb662d-92a6-452c-844b-b16042e3c618 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:29.252179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb662d-92a6-452c-844b-b16042e3c618 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:29.252274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb662d-92a6-452c-844b-b16042e3c618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:29.271487 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-cfdb662d-92a6-452c-844b-b16042e3c618 None None] MFA Rules not processed for user `79365a5e4ae2431d91a4aa603ba58fee`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:29:29.272690 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cfdb662d-92a6-452c-844b-b16042e3c618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:29.327172 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 998/2011] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:29:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:29:32.792996 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:29:33.042995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca1848-6e8a-4cfc-a9bc-0a31d8dd82f4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:33.043325 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca1848-6e8a-4cfc-a9bc-0a31d8dd82f4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:33.043586 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca1848-6e8a-4cfc-a9bc-0a31d8dd82f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:33.064559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d5ca1848-6e8a-4cfc-a9bc-0a31d8dd82f4 None None] MFA Rules not processed for user `e4ebab767c054b289e2279659a1cf2e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:29:33.066032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d5ca1848-6e8a-4cfc-a9bc-0a31d8dd82f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:33.135982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1014/2012] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:29:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:29:39.320574 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=79 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:29:40.065931 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb662d-92a6-452c-844b-b16042e3c618 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:40.067392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cfdb662d-92a6-452c-844b-b16042e3c618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:40.083534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb662d-92a6-452c-844b-b16042e3c618 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:40.089477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c235833-b797-41bb-bf4a-61749a0986eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:40.090077 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c235833-b797-41bb-bf4a-61749a0986eb service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:40.090356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c235833-b797-41bb-bf4a-61749a0986eb service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:40.090633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c235833-b797-41bb-bf4a-61749a0986eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:40.092309 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2c235833-b797-41bb-bf4a-61749a0986eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:40.102871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c235833-b797-41bb-bf4a-61749a0986eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:40.104618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c235833-b797-41bb-bf4a-61749a0986eb service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:40.113611 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 999/2013] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:29:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:29:43.128281 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=25 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:29:50.109398 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=6 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:29:56.795816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca1848-6e8a-4cfc-a9bc-0a31d8dd82f4 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:29:56.797389 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d5ca1848-6e8a-4cfc-a9bc-0a31d8dd82f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:56.811022 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca1848-6e8a-4cfc-a9bc-0a31d8dd82f4 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:29:56.814674 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed3240-83d6-4242-931a-0cb7924e6cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:29:56.815317 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed3240-83d6-4242-931a-0cb7924e6cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:29:56.815599 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed3240-83d6-4242-931a-0cb7924e6cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:29:56.815871 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed3240-83d6-4242-931a-0cb7924e6cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:29:56.817638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-baed3240-83d6-4242-931a-0cb7924e6cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:29:56.828464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed3240-83d6-4242-931a-0cb7924e6cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:29:56.830011 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed3240-83d6-4242-931a-0cb7924e6cd1 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:29:56.842408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1015/2014] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:29:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:30:03.221278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c235833-b797-41bb-bf4a-61749a0986eb service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:03.233457 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c235833-b797-41bb-bf4a-61749a0986eb service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:03.239243 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c7341-3092-474c-9006-53518c612533 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:03.239885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c7341-3092-474c-9006-53518c612533 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:03.240148 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c7341-3092-474c-9006-53518c612533 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:03.240419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c7341-3092-474c-9006-53518c612533 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:03.242116 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-249c7341-3092-474c-9006-53518c612533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:03.301245 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c7341-3092-474c-9006-53518c612533 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:03.302856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c7341-3092-474c-9006-53518c612533 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:03.312631 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1000/2015] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:30:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:30:06.838453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:11.337558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed3240-83d6-4242-931a-0cb7924e6cd1 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:11.339094 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-baed3240-83d6-4242-931a-0cb7924e6cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:11.356206 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed3240-83d6-4242-931a-0cb7924e6cd1 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:11.362104 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef1c2a-504d-41a9-a94d-f6d6f462a686 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:11.362724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef1c2a-504d-41a9-a94d-f6d6f462a686 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:11.363000 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef1c2a-504d-41a9-a94d-f6d6f462a686 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:11.363311 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef1c2a-504d-41a9-a94d-f6d6f462a686 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:11.365683 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d6ef1c2a-504d-41a9-a94d-f6d6f462a686 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:11.385297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef1c2a-504d-41a9-a94d-f6d6f462a686 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:11.386767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef1c2a-504d-41a9-a94d-f6d6f462a686 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:11.399467 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1016/2016] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:30:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:30:13.308385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=28 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:13.763231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c7341-3092-474c-9006-53518c612533 service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:13.764818 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-249c7341-3092-474c-9006-53518c612533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:13.827547 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c7341-3092-474c-9006-53518c612533 service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:13.831152 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884b3c1-8c8a-4076-9709-ff863655a991 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:13.831767 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b3c1-8c8a-4076-9709-ff863655a991 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:13.832045 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b3c1-8c8a-4076-9709-ff863655a991 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:13.832330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b3c1-8c8a-4076-9709-ff863655a991 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:13.834034 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3884b3c1-8c8a-4076-9709-ff863655a991 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:13.845043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884b3c1-8c8a-4076-9709-ff863655a991 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:13.846455 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884b3c1-8c8a-4076-9709-ff863655a991 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:13.855879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1001/2017] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:30:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:30:21.393778 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:22.859155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef1c2a-504d-41a9-a94d-f6d6f462a686 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:22.875553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef1c2a-504d-41a9-a94d-f6d6f462a686 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:22.894437 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ee015-d332-4868-a1e7-785723a2b096 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:22.900631 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ee015-d332-4868-a1e7-785723a2b096 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:22.901013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ee015-d332-4868-a1e7-785723a2b096 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:22.901290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ee015-d332-4868-a1e7-785723a2b096 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:22.903322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d09ee015-d332-4868-a1e7-785723a2b096 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:22.936242 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ee015-d332-4868-a1e7-785723a2b096 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:22.939375 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ee015-d332-4868-a1e7-785723a2b096 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:22.979577 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1017/2018] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:30:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:30:23.851287 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=26 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:25.167380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1b936-281a-4688-b345-a49e83e80d6b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:25.168040 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1b936-281a-4688-b345-a49e83e80d6b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:25.168040 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1b936-281a-4688-b345-a49e83e80d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:25.186825 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-14f1b936-281a-4688-b345-a49e83e80d6b None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:30:25.188051 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-14f1b936-281a-4688-b345-a49e83e80d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:25.246478 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1002/2019] 10.222.0.18 () {58 vars in 967 bytes} [Fri Oct 14 02:30:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:30:25.298732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ee015-d332-4868-a1e7-785723a2b096 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:25.300592 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d09ee015-d332-4868-a1e7-785723a2b096 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:25.377329 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ee015-d332-4868-a1e7-785723a2b096 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:25.384219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d10594-d20e-4fc0-854a-c22b80c73dd7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:25.385005 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d10594-d20e-4fc0-854a-c22b80c73dd7 service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:25.385294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d10594-d20e-4fc0-854a-c22b80c73dd7 service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:25.385568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d10594-d20e-4fc0-854a-c22b80c73dd7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:25.394859 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d10594-d20e-4fc0-854a-c22b80c73dd7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:25.396583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d10594-d20e-4fc0-854a-c22b80c73dd7 service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:25.405987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1018/2020] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:30:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:30:28.331653 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce98854-0fd7-4b8f-a742-471949bb6737 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:28.332166 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce98854-0fd7-4b8f-a742-471949bb6737 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:28.332438 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce98854-0fd7-4b8f-a742-471949bb6737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:28.360799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5ce98854-0fd7-4b8f-a742-471949bb6737 None None] MFA Rules not processed for user `7d4748cbe63d490b9d8b6914a27ff364`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:30:28.362271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5ce98854-0fd7-4b8f-a742-471949bb6737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:28.370191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1003/2021] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:30:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:30:29.172448 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d10594-d20e-4fc0-854a-c22b80c73dd7 service swift] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:29.188360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d10594-d20e-4fc0-854a-c22b80c73dd7 service swift] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:29.193006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2a7c0-031e-4595-8b4e-678699cc3882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:29.193899 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2a7c0-031e-4595-8b4e-678699cc3882 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:29.194524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2a7c0-031e-4595-8b4e-678699cc3882 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:29.195757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2a7c0-031e-4595-8b4e-678699cc3882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:29.197875 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-97e2a7c0-031e-4595-8b4e-678699cc3882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:29.225492 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2a7c0-031e-4595-8b4e-678699cc3882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:29.232342 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2a7c0-031e-4595-8b4e-678699cc3882 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:29.258263 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1019/2022] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:30:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:30:29.999308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce98854-0fd7-4b8f-a742-471949bb6737 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:30.001301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5ce98854-0fd7-4b8f-a742-471949bb6737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:30.185370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce98854-0fd7-4b8f-a742-471949bb6737 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:30.193549 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f7529-a944-4a12-9f53-1a7c0c35ce8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e935ba63cdd4256874b99a2eb0a1a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:30.197924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f7529-a944-4a12-9f53-1a7c0c35ce8f service placement] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:30.198326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f7529-a944-4a12-9f53-1a7c0c35ce8f service placement] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:30.198673 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f7529-a944-4a12-9f53-1a7c0c35ce8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:30.201035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6d5f7529-a944-4a12-9f53-1a7c0c35ce8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:30.243490 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f7529-a944-4a12-9f53-1a7c0c35ce8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:30.245431 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f7529-a944-4a12-9f53-1a7c0c35ce8f service placement] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:30.269151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1004/2023] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:30:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:30:36.712189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2a7c0-031e-4595-8b4e-678699cc3882 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:36.713731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-97e2a7c0-031e-4595-8b4e-678699cc3882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:36.804214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2a7c0-031e-4595-8b4e-678699cc3882 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:36.808780 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4dcba-211f-4524-8e26-481b2c3d2fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:36.809443 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4dcba-211f-4524-8e26-481b2c3d2fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:36.809718 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4dcba-211f-4524-8e26-481b2c3d2fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:36.809988 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4dcba-211f-4524-8e26-481b2c3d2fdf admin admin] PATH_INFO: `/v3/users/5e7267cf9bbf4bea81bc949221f31e18` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:36.811135 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4dcba-211f-4524-8e26-481b2c3d2fdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e7267cf9bbf4bea81bc949221f31e18)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:36.813056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4dcba-211f-4524-8e26-481b2c3d2fdf admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:36.862041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-22c4dcba-211f-4524-8e26-481b2c3d2fdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e7267cf9bbf4bea81bc949221f31e18'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:36.869428 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-22c4dcba-211f-4524-8e26-481b2c3d2fdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e7267cf9bbf4bea81bc949221f31e18'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:36.877266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:36.882892 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-22c4dcba-211f-4524-8e26-481b2c3d2fdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e7267cf9bbf4bea81bc949221f31e18'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:36.891230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-22c4dcba-211f-4524-8e26-481b2c3d2fdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e7267cf9bbf4bea81bc949221f31e18'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:36.891886 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-22c4dcba-211f-4524-8e26-481b2c3d2fdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e7267cf9bbf4bea81bc949221f31e18'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:36.897830 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1020/2024] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:30:36 2022] DELETE /identity/v3/users/5e7267cf9bbf4bea81bc949221f31e18 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:30:37.068030 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f7529-a944-4a12-9f53-1a7c0c35ce8f service placement] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:37.084560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f7529-a944-4a12-9f53-1a7c0c35ce8f service placement] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:37.142404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:37.143038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:37.143406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:37.143676 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] PATH_INFO: `/v3/projects/ace6580a1307486895a0ec8149a95fa9` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:37.144898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ace6580a1307486895a0ec8149a95fa9)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:37.146833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:37.184474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:37.199494 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ace6580a1307486895a0ec8149a95fa9', 'request_id': 'req-9c6f4e60-a620-4a8f-94b9-71269e542505'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.205361 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ace6580a1307486895a0ec8149a95fa9', 'request_id': 'req-9c6f4e60-a620-4a8f-94b9-71269e542505'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.211790 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ace6580a1307486895a0ec8149a95fa9', 'request_id': 'req-9c6f4e60-a620-4a8f-94b9-71269e542505'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.227018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] The token cache is being invalidate because project ace6580a1307486895a0ec8149a95fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:30:37.227343 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.227985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.229765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1005/2025] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:30:37 2022] DELETE /identity/v3/projects/ace6580a1307486895a0ec8149a95fa9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:30:37.233537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4dcba-211f-4524-8e26-481b2c3d2fdf admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:37.235271 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-22c4dcba-211f-4524-8e26-481b2c3d2fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:37.249966 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4dcba-211f-4524-8e26-481b2c3d2fdf admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:37.307962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143137dd-fdae-466a-bfa6-0a805bbbe2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:37.308595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143137dd-fdae-466a-bfa6-0a805bbbe2bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:37.308876 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143137dd-fdae-466a-bfa6-0a805bbbe2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:37.309150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143137dd-fdae-466a-bfa6-0a805bbbe2bc admin admin] PATH_INFO: `/v3/users/6fa96892d57d4d969e01d9014a50a8e9` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:37.310280 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143137dd-fdae-466a-bfa6-0a805bbbe2bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fa96892d57d4d969e01d9014a50a8e9)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:37.312247 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143137dd-fdae-466a-bfa6-0a805bbbe2bc admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:37.358294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-143137dd-fdae-466a-bfa6-0a805bbbe2bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fa96892d57d4d969e01d9014a50a8e9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.365591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-143137dd-fdae-466a-bfa6-0a805bbbe2bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fa96892d57d4d969e01d9014a50a8e9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.378135 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-143137dd-fdae-466a-bfa6-0a805bbbe2bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fa96892d57d4d969e01d9014a50a8e9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.385351 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-143137dd-fdae-466a-bfa6-0a805bbbe2bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fa96892d57d4d969e01d9014a50a8e9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.386012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-143137dd-fdae-466a-bfa6-0a805bbbe2bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fa96892d57d4d969e01d9014a50a8e9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.391771 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1021/2026] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:30:37 2022] DELETE /identity/v3/users/6fa96892d57d4d969e01d9014a50a8e9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:30:37.565143 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:37.566670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:37.581450 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f4e60-a620-4a8f-94b9-71269e542505 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:37.668951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6f997-74ab-42c1-865a-b549099bf7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:37.669779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6f997-74ab-42c1-865a-b549099bf7de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:37.670129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6f997-74ab-42c1-865a-b549099bf7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:37.670402 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6f997-74ab-42c1-865a-b549099bf7de admin admin] PATH_INFO: `/v3/projects/45d54683464248b08141b7fe35047ecb` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:37.671563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6f997-74ab-42c1-865a-b549099bf7de admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d54683464248b08141b7fe35047ecb)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:37.673478 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6f997-74ab-42c1-865a-b549099bf7de admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:37.721894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d4e6f997-74ab-42c1-865a-b549099bf7de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d54683464248b08141b7fe35047ecb', 'request_id': 'req-d4e6f997-74ab-42c1-865a-b549099bf7de'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.735685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d4e6f997-74ab-42c1-865a-b549099bf7de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d54683464248b08141b7fe35047ecb', 'request_id': 'req-d4e6f997-74ab-42c1-865a-b549099bf7de'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.746876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d4e6f997-74ab-42c1-865a-b549099bf7de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d54683464248b08141b7fe35047ecb', 'request_id': 'req-d4e6f997-74ab-42c1-865a-b549099bf7de'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.762625 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d4e6f997-74ab-42c1-865a-b549099bf7de admin admin] The token cache is being invalidate because project 45d54683464248b08141b7fe35047ecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:30:37.762917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d4e6f997-74ab-42c1-865a-b549099bf7de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.763616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d4e6f997-74ab-42c1-865a-b549099bf7de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:37.765162 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1006/2027] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:30:37 2022] DELETE /identity/v3/projects/45d54683464248b08141b7fe35047ecb => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:30:37.770137 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06983ce-3192-48cd-9fb2-62ffe3a0e1a5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:37.770617 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06983ce-3192-48cd-9fb2-62ffe3a0e1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:37.770718 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06983ce-3192-48cd-9fb2-62ffe3a0e1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:37.792688 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e06983ce-3192-48cd-9fb2-62ffe3a0e1a5 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:30:37.793884 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e06983ce-3192-48cd-9fb2-62ffe3a0e1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:37.858171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1022/2028] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:30:37.866693 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b5281-3ae2-40bf-984a-20f2e6a95018 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:37.866979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b5281-3ae2-40bf-984a-20f2e6a95018 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:37.867296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b5281-3ae2-40bf-984a-20f2e6a95018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:37.886829 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-404b5281-3ae2-40bf-984a-20f2e6a95018 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:30:37.888059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-404b5281-3ae2-40bf-984a-20f2e6a95018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:37.895913 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1007/2029] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:30:37.900737 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06983ce-3192-48cd-9fb2-62ffe3a0e1a5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:37.905568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:37.910838 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06983ce-3192-48cd-9fb2-62ffe3a0e1a5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:37.920724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34c516-7d06-4480-ab95-17c4be45f243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:37.921531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34c516-7d06-4480-ab95-17c4be45f243 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:37.921879 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34c516-7d06-4480-ab95-17c4be45f243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:37.922180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34c516-7d06-4480-ab95-17c4be45f243 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:37.922877 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34c516-7d06-4480-ab95-17c4be45f243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:30:37.923221 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34c516-7d06-4480-ab95-17c4be45f243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:37.924890 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34c516-7d06-4480-ab95-17c4be45f243 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:37.940244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1023/2030] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:30:37.941286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b5281-3ae2-40bf-984a-20f2e6a95018 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:37.950503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b5281-3ae2-40bf-984a-20f2e6a95018 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:37.955144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad77927-6971-42cd-ba13-5f1ac2a2f655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:37.955763 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad77927-6971-42cd-ba13-5f1ac2a2f655 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:37.956044 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad77927-6971-42cd-ba13-5f1ac2a2f655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:37.956422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad77927-6971-42cd-ba13-5f1ac2a2f655 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:37.957059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad77927-6971-42cd-ba13-5f1ac2a2f655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:37.958963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad77927-6971-42cd-ba13-5f1ac2a2f655 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:37.990838 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1008/2031] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:30:37 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:30:37.992795 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34c516-7d06-4480-ab95-17c4be45f243 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:38.003324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34c516-7d06-4480-ab95-17c4be45f243 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:38.081725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f06e8-0a32-410e-a4ba-89b09bd8267d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:38.082459 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f06e8-0a32-410e-a4ba-89b09bd8267d admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:38.082737 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f06e8-0a32-410e-a4ba-89b09bd8267d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:38.083009 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f06e8-0a32-410e-a4ba-89b09bd8267d admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:38.083683 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f06e8-0a32-410e-a4ba-89b09bd8267d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:38.086472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f06e8-0a32-410e-a4ba-89b09bd8267d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:38.108391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1024/2032] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:30:37 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:30:38.112964 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad77927-6971-42cd-ba13-5f1ac2a2f655 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:38.123280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad77927-6971-42cd-ba13-5f1ac2a2f655 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:38.128963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16974436-d8a6-45fd-9c59-761b0d02a665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:38.129390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16974436-d8a6-45fd-9c59-761b0d02a665 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:38.129668 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16974436-d8a6-45fd-9c59-761b0d02a665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:38.129938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16974436-d8a6-45fd-9c59-761b0d02a665 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:38.130578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16974436-d8a6-45fd-9c59-761b0d02a665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:38.134401 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16974436-d8a6-45fd-9c59-761b0d02a665 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:38.144906 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1009/2033] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:30:38.149604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f06e8-0a32-410e-a4ba-89b09bd8267d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:38.160054 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f06e8-0a32-410e-a4ba-89b09bd8267d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:38.166123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7641e-39bf-43fc-bc5e-c7434039fa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:38.166723 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7641e-39bf-43fc-bc5e-c7434039fa20 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:38.167003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7641e-39bf-43fc-bc5e-c7434039fa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:38.167333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7641e-39bf-43fc-bc5e-c7434039fa20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:38.167998 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7641e-39bf-43fc-bc5e-c7434039fa20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:38.169599 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7641e-39bf-43fc-bc5e-c7434039fa20 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:38.180488 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1025/2034] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:30:38.184653 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16974436-d8a6-45fd-9c59-761b0d02a665 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:38.195172 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16974436-d8a6-45fd-9c59-761b0d02a665 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:38.200154 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ea589-75ff-4d42-963e-5fd73f0ac679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:38.200938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ea589-75ff-4d42-963e-5fd73f0ac679 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:38.201219 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ea589-75ff-4d42-963e-5fd73f0ac679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:38.201493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ea589-75ff-4d42-963e-5fd73f0ac679 admin admin] PATH_INFO: `/v3/projects/3d1ba99898ca42c19c8f48fa498c5851/users/eca98759b55b443b9591eb6c80501157/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:38.204531 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:38.218440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ea589-75ff-4d42-963e-5fd73f0ac679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1ba99898ca42c19c8f48fa498c5851, user_id=eca98759b55b443b9591eb6c80501157, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:38.221143 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ea589-75ff-4d42-963e-5fd73f0ac679 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:38.235861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1010/2035] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:30:38 2022] PUT /identity/v3/projects/3d1ba99898ca42c19c8f48fa498c5851/users/eca98759b55b443b9591eb6c80501157/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:30:38.240634 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbe9e1-07f0-4dca-a33a-191f58d701b9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:38.240992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbe9e1-07f0-4dca-a33a-191f58d701b9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:38.241336 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbe9e1-07f0-4dca-a33a-191f58d701b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:38.260747 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-61cbe9e1-07f0-4dca-a33a-191f58d701b9 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:30:38.262000 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-61cbe9e1-07f0-4dca-a33a-191f58d701b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:38.346250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1026/2036] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:30:38.365590 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ea589-75ff-4d42-963e-5fd73f0ac679 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:38.365590 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d20ea589-75ff-4d42-963e-5fd73f0ac679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:38.368061 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbe9e1-07f0-4dca-a33a-191f58d701b9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:38.373601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-61cbe9e1-07f0-4dca-a33a-191f58d701b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:38.472823 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ea589-75ff-4d42-963e-5fd73f0ac679 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:38.478517 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f41db-bc2e-41c6-87c6-6045568da326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:38.478636 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f41db-bc2e-41c6-87c6-6045568da326 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:38.478912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f41db-bc2e-41c6-87c6-6045568da326 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:38.479247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f41db-bc2e-41c6-87c6-6045568da326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:38.482401 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbe9e1-07f0-4dca-a33a-191f58d701b9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:38.487563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88bb05-43c4-46e2-a6a1-dc99a3b382c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:38.488535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88bb05-43c4-46e2-a6a1-dc99a3b382c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:38.488535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88bb05-43c4-46e2-a6a1-dc99a3b382c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:38.488731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88bb05-43c4-46e2-a6a1-dc99a3b382c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:38.490444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6e88bb05-43c4-46e2-a6a1-dc99a3b382c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:38.500919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f41db-bc2e-41c6-87c6-6045568da326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:38.502356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f41db-bc2e-41c6-87c6-6045568da326 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:38.509028 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88bb05-43c4-46e2-a6a1-dc99a3b382c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:38.510472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88bb05-43c4-46e2-a6a1-dc99a3b382c1 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:38.513200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1011/2037] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:30:38.525120 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1027/2038] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:30:44.224578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f41db-bc2e-41c6-87c6-6045568da326 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:44.230525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a74f41db-bc2e-41c6-87c6-6045568da326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:44.250041 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f41db-bc2e-41c6-87c6-6045568da326 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:44.276469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a050e-4baa-4e40-b771-523ddb0afb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:44.277116 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a050e-4baa-4e40-b771-523ddb0afb85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:44.278589 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a050e-4baa-4e40-b771-523ddb0afb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:44.279874 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a050e-4baa-4e40-b771-523ddb0afb85 admin admin] PATH_INFO: `/v3/users/79365a5e4ae2431d91a4aa603ba58fee` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:44.282019 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a050e-4baa-4e40-b771-523ddb0afb85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79365a5e4ae2431d91a4aa603ba58fee)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:44.284178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a050e-4baa-4e40-b771-523ddb0afb85 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:44.427983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88bb05-43c4-46e2-a6a1-dc99a3b382c1 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:44.430501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6e88bb05-43c4-46e2-a6a1-dc99a3b382c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:44.444687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e43a050e-4baa-4e40-b771-523ddb0afb85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79365a5e4ae2431d91a4aa603ba58fee'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:44.457723 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e43a050e-4baa-4e40-b771-523ddb0afb85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79365a5e4ae2431d91a4aa603ba58fee'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:44.459426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e43a050e-4baa-4e40-b771-523ddb0afb85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79365a5e4ae2431d91a4aa603ba58fee'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:44.467209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:44.514036 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e43a050e-4baa-4e40-b771-523ddb0afb85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79365a5e4ae2431d91a4aa603ba58fee'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:44.540323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e43a050e-4baa-4e40-b771-523ddb0afb85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79365a5e4ae2431d91a4aa603ba58fee'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:44.562650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1012/2039] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:30:44 2022] DELETE /identity/v3/users/79365a5e4ae2431d91a4aa603ba58fee => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:30:44.657713 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88bb05-43c4-46e2-a6a1-dc99a3b382c1 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:44.789771 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5919bf-99c5-44e7-80db-ddc7878bb6ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:44.790577 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5919bf-99c5-44e7-80db-ddc7878bb6ad service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:44.790864 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5919bf-99c5-44e7-80db-ddc7878bb6ad service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:44.791197 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5919bf-99c5-44e7-80db-ddc7878bb6ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:44.799469 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7b5919bf-99c5-44e7-80db-ddc7878bb6ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:44.828358 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5919bf-99c5-44e7-80db-ddc7878bb6ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:44.834737 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5919bf-99c5-44e7-80db-ddc7878bb6ad service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:44.861433 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1028/2040] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:30:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:30:44.980113 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a050e-4baa-4e40-b771-523ddb0afb85 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:44.982400 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e43a050e-4baa-4e40-b771-523ddb0afb85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:45.117385 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a050e-4baa-4e40-b771-523ddb0afb85 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:45.125091 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:45.126723 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:45.127323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:45.127435 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2 admin admin] PATH_INFO: `/v3/projects/a2e38a12e174437982573067f6bc3f8d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:45.128753 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2e38a12e174437982573067f6bc3f8d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:45.130742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:45.134147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:45.220597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2e38a12e174437982573067f6bc3f8d', 'request_id': 'req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:45.227683 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2e38a12e174437982573067f6bc3f8d', 'request_id': 'req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:45.237355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2e38a12e174437982573067f6bc3f8d', 'request_id': 'req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:45.260006 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2 admin admin] The token cache is being invalidate because project a2e38a12e174437982573067f6bc3f8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:30:45.260673 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:45.261706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f237b0bc-290a-4a14-83bb-aa5df8aa2cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:30:45.268907 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1013/2041] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:30:44 2022] DELETE /identity/v3/projects/a2e38a12e174437982573067f6bc3f8d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:30:45.274276 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465634e4-76a3-477b-b290-58bafa64645c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:45.274578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465634e4-76a3-477b-b290-58bafa64645c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:45.274850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465634e4-76a3-477b-b290-58bafa64645c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:45.318145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-465634e4-76a3-477b-b290-58bafa64645c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:30:45.321273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-465634e4-76a3-477b-b290-58bafa64645c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:45.464971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1029/2042] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:30:45.468534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476f76c-47ae-4d54-b337-cd2973e5980c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:45.468534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476f76c-47ae-4d54-b337-cd2973e5980c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:45.468895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476f76c-47ae-4d54-b337-cd2973e5980c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:45.508343 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-3476f76c-47ae-4d54-b337-cd2973e5980c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:30:45.510380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3476f76c-47ae-4d54-b337-cd2973e5980c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:45.527421 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1014/2043] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:30:45.528223 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465634e4-76a3-477b-b290-58bafa64645c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:45.540946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465634e4-76a3-477b-b290-58bafa64645c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:45.549841 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fecd5-e94b-4e99-87a9-cdc75d2476c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:45.550609 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fecd5-e94b-4e99-87a9-cdc75d2476c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:45.552012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fecd5-e94b-4e99-87a9-cdc75d2476c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:45.552012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fecd5-e94b-4e99-87a9-cdc75d2476c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:45.554370 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fecd5-e94b-4e99-87a9-cdc75d2476c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:30:45.555034 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fecd5-e94b-4e99-87a9-cdc75d2476c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:45.556980 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fecd5-e94b-4e99-87a9-cdc75d2476c1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:45.581156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476f76c-47ae-4d54-b337-cd2973e5980c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:45.585714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1030/2044] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:30:45.591648 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476f76c-47ae-4d54-b337-cd2973e5980c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:45.596838 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d95dcb-8837-4c59-a208-f848cc5ee811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:45.597577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d95dcb-8837-4c59-a208-f848cc5ee811 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:45.597980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d95dcb-8837-4c59-a208-f848cc5ee811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:45.598465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d95dcb-8837-4c59-a208-f848cc5ee811 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:45.599463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d95dcb-8837-4c59-a208-f848cc5ee811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:45.603048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d95dcb-8837-4c59-a208-f848cc5ee811 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:45.660995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fecd5-e94b-4e99-87a9-cdc75d2476c1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:45.663986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1015/2045] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:30:45 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:30:45.695997 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fecd5-e94b-4e99-87a9-cdc75d2476c1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:45.763642 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:45.840844 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc48cc-0d98-437e-a41c-32d6d85fcf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:45.841869 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc48cc-0d98-437e-a41c-32d6d85fcf30 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:45.842354 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc48cc-0d98-437e-a41c-32d6d85fcf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:45.842905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc48cc-0d98-437e-a41c-32d6d85fcf30 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:45.843681 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc48cc-0d98-437e-a41c-32d6d85fcf30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:45.845731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc48cc-0d98-437e-a41c-32d6d85fcf30 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:45.887852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1031/2046] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:30:45 2022] POST /identity/v3/users => generated 286 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:30:45.892167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d95dcb-8837-4c59-a208-f848cc5ee811 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:45.912702 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d95dcb-8837-4c59-a208-f848cc5ee811 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:45.919900 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccea6c-b200-447d-af98-ab7a54fcfb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:45.922458 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccea6c-b200-447d-af98-ab7a54fcfb12 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:45.923122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccea6c-b200-447d-af98-ab7a54fcfb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:45.924817 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccea6c-b200-447d-af98-ab7a54fcfb12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:45.926093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccea6c-b200-447d-af98-ab7a54fcfb12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:45.930628 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccea6c-b200-447d-af98-ab7a54fcfb12 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:45.947707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1016/2047] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:30:45.951605 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc48cc-0d98-437e-a41c-32d6d85fcf30 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:45.988487 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc48cc-0d98-437e-a41c-32d6d85fcf30 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:46.002073 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a5371-5cde-40d0-af0f-5d733419e9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:46.002931 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5371-5cde-40d0-af0f-5d733419e9ea admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:46.003051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5371-5cde-40d0-af0f-5d733419e9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:46.003319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5371-5cde-40d0-af0f-5d733419e9ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:46.011188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a5371-5cde-40d0-af0f-5d733419e9ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:46.011188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a5371-5cde-40d0-af0f-5d733419e9ea admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:46.032577 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1032/2048] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:30:46.037052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccea6c-b200-447d-af98-ab7a54fcfb12 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:46.052531 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccea6c-b200-447d-af98-ab7a54fcfb12 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:46.078231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4297c-c49c-4e5d-baa6-3532eb5cd18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:46.079596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4297c-c49c-4e5d-baa6-3532eb5cd18e admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:46.080704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4297c-c49c-4e5d-baa6-3532eb5cd18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:46.081524 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4297c-c49c-4e5d-baa6-3532eb5cd18e admin admin] PATH_INFO: `/v3/projects/220eedf0de8b4b7dbaff769bd12af44a/users/15b55f42d0554e11904da66af9645dbc/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:46.124597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4297c-c49c-4e5d-baa6-3532eb5cd18e admin admin] RBAC: Authorizing `identity:create_grant(project_id=220eedf0de8b4b7dbaff769bd12af44a, user_id=15b55f42d0554e11904da66af9645dbc, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:46.129538 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4297c-c49c-4e5d-baa6-3532eb5cd18e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:46.150916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1017/2049] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:30:46 2022] PUT /identity/v3/projects/220eedf0de8b4b7dbaff769bd12af44a/users/15b55f42d0554e11904da66af9645dbc/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:30:46.169987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea01f-8a5f-4c97-adb8-595740951e82 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:46.170271 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea01f-8a5f-4c97-adb8-595740951e82 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:46.170640 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea01f-8a5f-4c97-adb8-595740951e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:46.217818 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9dbea01f-8a5f-4c97-adb8-595740951e82 None None] MFA Rules not processed for user `15b55f42d0554e11904da66af9645dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:30:46.219735 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9dbea01f-8a5f-4c97-adb8-595740951e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:46.382608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1033/2050] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:30:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:30:46.388398 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4297c-c49c-4e5d-baa6-3532eb5cd18e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:46.555634 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4297c-c49c-4e5d-baa6-3532eb5cd18e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:46.583061 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb1c30-5b1b-4012-8919-f64a9c4347b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:46.584419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb1c30-5b1b-4012-8919-f64a9c4347b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:46.584419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb1c30-5b1b-4012-8919-f64a9c4347b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:46.588394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb1c30-5b1b-4012-8919-f64a9c4347b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:46.589172 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb1c30-5b1b-4012-8919-f64a9c4347b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:46.594918 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb1c30-5b1b-4012-8919-f64a9c4347b2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:46.663037 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1018/2051] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:30:46 2022] POST /identity/v3/projects => generated 373 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:30:46.668363 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea01f-8a5f-4c97-adb8-595740951e82 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:46.693461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea01f-8a5f-4c97-adb8-595740951e82 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:46.879796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfea057-2007-4f61-bbec-d4bb5259e6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:46.880435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfea057-2007-4f61-bbec-d4bb5259e6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:46.880718 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfea057-2007-4f61-bbec-d4bb5259e6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:46.880994 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfea057-2007-4f61-bbec-d4bb5259e6f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:46.881887 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfea057-2007-4f61-bbec-d4bb5259e6f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:46.889982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfea057-2007-4f61-bbec-d4bb5259e6f6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:46.935604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1034/2052] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:30:46 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:30:46.939518 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb1c30-5b1b-4012-8919-f64a9c4347b2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:46.967667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb1c30-5b1b-4012-8919-f64a9c4347b2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:46.985201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf42a6-5ca1-40d1-8491-cf86be0441e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:46.986109 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf42a6-5ca1-40d1-8491-cf86be0441e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:46.986543 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf42a6-5ca1-40d1-8491-cf86be0441e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:46.986996 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf42a6-5ca1-40d1-8491-cf86be0441e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:46.987843 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf42a6-5ca1-40d1-8491-cf86be0441e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:46.989637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf42a6-5ca1-40d1-8491-cf86be0441e5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:47.005482 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:47.034430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1019/2053] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:30:47.035170 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfea057-2007-4f61-bbec-d4bb5259e6f6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:47.056328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfea057-2007-4f61-bbec-d4bb5259e6f6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:47.080340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaff7ff-2e70-463d-ac54-9fda98ab1e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:47.083725 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaff7ff-2e70-463d-ac54-9fda98ab1e8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:47.084333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaff7ff-2e70-463d-ac54-9fda98ab1e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:47.084966 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaff7ff-2e70-463d-ac54-9fda98ab1e8f admin admin] PATH_INFO: `/v3/projects/487dc0aefb3645fba15de26d96ec0a7d/users/aa84093f23f54eeca6770a3498340d5e/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:47.123943 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaff7ff-2e70-463d-ac54-9fda98ab1e8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=487dc0aefb3645fba15de26d96ec0a7d, user_id=aa84093f23f54eeca6770a3498340d5e, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:47.128772 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaff7ff-2e70-463d-ac54-9fda98ab1e8f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:47.132360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:47.150009 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1035/2054] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:30:47 2022] PUT /identity/v3/projects/487dc0aefb3645fba15de26d96ec0a7d/users/aa84093f23f54eeca6770a3498340d5e/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:30:47.154068 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf42a6-5ca1-40d1-8491-cf86be0441e5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:47.168389 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf42a6-5ca1-40d1-8491-cf86be0441e5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:47.298553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87eee83-dcb2-415c-9325-45ea36a02668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:47.299230 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87eee83-dcb2-415c-9325-45ea36a02668 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:47.299510 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87eee83-dcb2-415c-9325-45ea36a02668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:47.299782 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87eee83-dcb2-415c-9325-45ea36a02668 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:47.300425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87eee83-dcb2-415c-9325-45ea36a02668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:47.302741 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87eee83-dcb2-415c-9325-45ea36a02668 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:47.325127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1020/2055] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:30:47.329397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaff7ff-2e70-463d-ac54-9fda98ab1e8f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:47.352860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaff7ff-2e70-463d-ac54-9fda98ab1e8f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:47.361809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b240360-1069-43b0-83e9-55caa41a865e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:47.367865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b240360-1069-43b0-83e9-55caa41a865e admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:47.368366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b240360-1069-43b0-83e9-55caa41a865e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:47.369873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b240360-1069-43b0-83e9-55caa41a865e admin admin] PATH_INFO: `/v3/projects/487dc0aefb3645fba15de26d96ec0a7d/users/aa84093f23f54eeca6770a3498340d5e/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:47.371325 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b240360-1069-43b0-83e9-55caa41a865e admin admin] RBAC: Authorizing `identity:create_grant(project_id=487dc0aefb3645fba15de26d96ec0a7d, user_id=aa84093f23f54eeca6770a3498340d5e, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:47.374757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b240360-1069-43b0-83e9-55caa41a865e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:47.407632 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1036/2056] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:30:47 2022] PUT /identity/v3/projects/487dc0aefb3645fba15de26d96ec0a7d/users/aa84093f23f54eeca6770a3498340d5e/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:30:47.414484 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa069f-f4a2-430f-9f09-7fa91f0b24fd None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:47.415270 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa069f-f4a2-430f-9f09-7fa91f0b24fd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:47.415555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa069f-f4a2-430f-9f09-7fa91f0b24fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:47.557814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-dffa069f-f4a2-430f-9f09-7fa91f0b24fd None None] MFA Rules not processed for user `aa84093f23f54eeca6770a3498340d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:30:47.560303 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dffa069f-f4a2-430f-9f09-7fa91f0b24fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:47.768425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1021/2057] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:30:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:30:47.784353 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b240360-1069-43b0-83e9-55caa41a865e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:47.790465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4b240360-1069-43b0-83e9-55caa41a865e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:47.913187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b240360-1069-43b0-83e9-55caa41a865e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:47.927322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805fc8b1-3ef4-4f1d-ba97-07f99cb27593 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:47.928686 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fc8b1-3ef4-4f1d-ba97-07f99cb27593 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:47.929127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fc8b1-3ef4-4f1d-ba97-07f99cb27593 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:47.929537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fc8b1-3ef4-4f1d-ba97-07f99cb27593 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:47.954486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805fc8b1-3ef4-4f1d-ba97-07f99cb27593 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:47.956010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805fc8b1-3ef4-4f1d-ba97-07f99cb27593 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:47.980316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1037/2058] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:30:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:30:48.131552 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa069f-f4a2-430f-9f09-7fa91f0b24fd None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:48.147049 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa069f-f4a2-430f-9f09-7fa91f0b24fd None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:48.155769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69b97e-f648-4ba0-83f2-2182f2e3099e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:48.156423 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69b97e-f648-4ba0-83f2-2182f2e3099e service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:48.156740 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69b97e-f648-4ba0-83f2-2182f2e3099e service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:48.158882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69b97e-f648-4ba0-83f2-2182f2e3099e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:48.270248 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69b97e-f648-4ba0-83f2-2182f2e3099e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:48.274871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69b97e-f648-4ba0-83f2-2182f2e3099e service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:48.298226 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1022/2059] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:30:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:30:50.763655 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff8f04-c06f-4956-9b64-53346b5c422b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:50.766394 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff8f04-c06f-4956-9b64-53346b5c422b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:50.766905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff8f04-c06f-4956-9b64-53346b5c422b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:50.830184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-59ff8f04-c06f-4956-9b64-53346b5c422b None None] MFA Rules not processed for user `eca98759b55b443b9591eb6c80501157`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:30:50.831637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-59ff8f04-c06f-4956-9b64-53346b5c422b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:51.040472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1038/2060] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:30:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:30:51.047479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69b97e-f648-4ba0-83f2-2182f2e3099e service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:51.054562 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bf69b97e-f648-4ba0-83f2-2182f2e3099e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:51.105913 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:51.225920 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69b97e-f648-4ba0-83f2-2182f2e3099e service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:51.249898 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b6372-19a5-4702-bd07-9b1e6bf68af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:51.250786 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b6372-19a5-4702-bd07-9b1e6bf68af7 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:51.251090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b6372-19a5-4702-bd07-9b1e6bf68af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:51.251358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b6372-19a5-4702-bd07-9b1e6bf68af7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:51.253196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b6372-19a5-4702-bd07-9b1e6bf68af7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:51.256453 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b6372-19a5-4702-bd07-9b1e6bf68af7 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:51.309282 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1023/2061] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:30:51 2022] POST /identity/v3/projects => generated 377 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:30:51.310209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff8f04-c06f-4956-9b64-53346b5c422b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:51.335029 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff8f04-c06f-4956-9b64-53346b5c422b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:51.503787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1b002-a2af-48eb-abbe-4141fcedae81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:51.504421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1b002-a2af-48eb-abbe-4141fcedae81 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:51.504707 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1b002-a2af-48eb-abbe-4141fcedae81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:51.504986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1b002-a2af-48eb-abbe-4141fcedae81 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:51.506079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca1b002-a2af-48eb-abbe-4141fcedae81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:51.520958 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca1b002-a2af-48eb-abbe-4141fcedae81 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:51.590078 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b6372-19a5-4702-bd07-9b1e6bf68af7 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:51.591437 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1039/2062] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:30:51 2022] POST /identity/v3/users => generated 295 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:30:51.618214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b6372-19a5-4702-bd07-9b1e6bf68af7 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:51.640342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1f93e-245e-49e5-8059-4ffbab443131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:51.641993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1f93e-245e-49e5-8059-4ffbab443131 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:51.641993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1f93e-245e-49e5-8059-4ffbab443131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:51.641993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1f93e-245e-49e5-8059-4ffbab443131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:51.642993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1f93e-245e-49e5-8059-4ffbab443131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:51.646386 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1f93e-245e-49e5-8059-4ffbab443131 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:51.683800 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1024/2063] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:30:51.685017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1b002-a2af-48eb-abbe-4141fcedae81 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:51.692083 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:51.708503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1b002-a2af-48eb-abbe-4141fcedae81 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:51.725542 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058a0d1-0518-4179-accf-7f8734ce9f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:51.725542 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058a0d1-0518-4179-accf-7f8734ce9f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:51.725542 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058a0d1-0518-4179-accf-7f8734ce9f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:51.725866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058a0d1-0518-4179-accf-7f8734ce9f1b admin admin] PATH_INFO: `/v3/projects/a3033337c7734a46a7c933fbf5325521/users/127a44eadc72477a984614cc29102512/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:51.781838 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058a0d1-0518-4179-accf-7f8734ce9f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3033337c7734a46a7c933fbf5325521, user_id=127a44eadc72477a984614cc29102512, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:51.800013 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058a0d1-0518-4179-accf-7f8734ce9f1b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:51.841061 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1040/2064] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:30:51 2022] PUT /identity/v3/projects/a3033337c7734a46a7c933fbf5325521/users/127a44eadc72477a984614cc29102512/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:30:51.845181 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1f93e-245e-49e5-8059-4ffbab443131 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:51.861169 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1f93e-245e-49e5-8059-4ffbab443131 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:52.044599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d780c3f-a078-4732-aa57-92f29146b036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:52.046547 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d780c3f-a078-4732-aa57-92f29146b036 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:52.049444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d780c3f-a078-4732-aa57-92f29146b036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:52.050029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d780c3f-a078-4732-aa57-92f29146b036 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:52.055726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d780c3f-a078-4732-aa57-92f29146b036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:52.063804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d780c3f-a078-4732-aa57-92f29146b036 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:52.133144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058a0d1-0518-4179-accf-7f8734ce9f1b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:52.136432 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1025/2065] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:30:52.172088 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058a0d1-0518-4179-accf-7f8734ce9f1b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:52.201403 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef85695-7012-49dd-880e-47c2d4139086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:52.205885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef85695-7012-49dd-880e-47c2d4139086 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:52.208553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef85695-7012-49dd-880e-47c2d4139086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:52.209112 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef85695-7012-49dd-880e-47c2d4139086 admin admin] PATH_INFO: `/v3/projects/a3033337c7734a46a7c933fbf5325521/users/127a44eadc72477a984614cc29102512/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:52.220912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef85695-7012-49dd-880e-47c2d4139086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3033337c7734a46a7c933fbf5325521, user_id=127a44eadc72477a984614cc29102512, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:52.224998 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef85695-7012-49dd-880e-47c2d4139086 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:52.245264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1041/2066] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:30:52 2022] PUT /identity/v3/projects/a3033337c7734a46a7c933fbf5325521/users/127a44eadc72477a984614cc29102512/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:30:52.418762 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b06d0-766c-433c-8bd8-f4951288db5c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:52.418955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b06d0-766c-433c-8bd8-f4951288db5c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:52.419551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b06d0-766c-433c-8bd8-f4951288db5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:52.493523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a24b06d0-766c-433c-8bd8-f4951288db5c None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:30:52.499522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a24b06d0-766c-433c-8bd8-f4951288db5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:52.642167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1026/2067] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:30:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:30:52.661611 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef85695-7012-49dd-880e-47c2d4139086 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:52.663356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fef85695-7012-49dd-880e-47c2d4139086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:52.837712 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef85695-7012-49dd-880e-47c2d4139086 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:52.844907 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da0f99-2cc8-40cd-b617-f78d85735ca0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:52.846316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da0f99-2cc8-40cd-b617-f78d85735ca0 service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:52.846751 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da0f99-2cc8-40cd-b617-f78d85735ca0 service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:52.847214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da0f99-2cc8-40cd-b617-f78d85735ca0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:52.860801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8da0f99-2cc8-40cd-b617-f78d85735ca0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:52.863039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8da0f99-2cc8-40cd-b617-f78d85735ca0 service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:52.890258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1042/2068] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:30:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:30:54.979139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2c44d-269f-4c16-91c1-aa75eceec4a4 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:54.979660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2c44d-269f-4c16-91c1-aa75eceec4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:54.981983 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2c44d-269f-4c16-91c1-aa75eceec4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:55.014519 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d2b2c44d-269f-4c16-91c1-aa75eceec4a4 None None] MFA Rules not processed for user `d9f9821cd90e4d9096cc01aee5c0e569`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:30:55.015709 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d2b2c44d-269f-4c16-91c1-aa75eceec4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:55.057714 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:30:55.127584 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1027/2069] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:30:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:30:56.122541 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da0f99-2cc8-40cd-b617-f78d85735ca0 service swift] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:30:56.124001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b8da0f99-2cc8-40cd-b617-f78d85735ca0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:56.267886 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da0f99-2cc8-40cd-b617-f78d85735ca0 service swift] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:30:56.278018 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c789d-8311-42a8-b425-aff241c993d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:30:56.280505 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c789d-8311-42a8-b425-aff241c993d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:30:56.281009 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c789d-8311-42a8-b425-aff241c993d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:30:56.281512 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c789d-8311-42a8-b425-aff241c993d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:30:56.290919 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2e8c789d-8311-42a8-b425-aff241c993d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:30:56.317051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c789d-8311-42a8-b425-aff241c993d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:30:56.320167 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c789d-8311-42a8-b425-aff241c993d5 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:30:56.330911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1043/2070] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:30:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:31:02.461628 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52502dd-881f-4ee5-8bfe-1e13c8987fd7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:02.462105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52502dd-881f-4ee5-8bfe-1e13c8987fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:02.462105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52502dd-881f-4ee5-8bfe-1e13c8987fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:02.503621 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f52502dd-881f-4ee5-8bfe-1e13c8987fd7 None None] MFA Rules not processed for user `127a44eadc72477a984614cc29102512`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:02.505017 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f52502dd-881f-4ee5-8bfe-1e13c8987fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:02.686400 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1028/2071] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:02.698111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c789d-8311-42a8-b425-aff241c993d5 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:02.699587 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2e8c789d-8311-42a8-b425-aff241c993d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:02.831850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:02.858330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c789d-8311-42a8-b425-aff241c993d5 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:02.865561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356102a7-2dc4-4733-b923-18a50691e936 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:02.866816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356102a7-2dc4-4733-b923-18a50691e936 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:02.867374 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356102a7-2dc4-4733-b923-18a50691e936 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:02.867865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356102a7-2dc4-4733-b923-18a50691e936 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:03.019220 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356102a7-2dc4-4733-b923-18a50691e936 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:03.026558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356102a7-2dc4-4733-b923-18a50691e936 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:03.079751 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1044/2072] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:31:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:31:12.670926 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=43 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:13.054776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=29 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:21.420486 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047b6c1-d41e-4ff7-bf16-7d6aa8fbcfbc None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:21.420960 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047b6c1-d41e-4ff7-bf16-7d6aa8fbcfbc None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:21.421052 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047b6c1-d41e-4ff7-bf16-7d6aa8fbcfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:21.440147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b047b6c1-d41e-4ff7-bf16-7d6aa8fbcfbc None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:21.441418 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b047b6c1-d41e-4ff7-bf16-7d6aa8fbcfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:21.449224 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1029/2073] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:31:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:21.461738 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356102a7-2dc4-4733-b923-18a50691e936 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:21.472698 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356102a7-2dc4-4733-b923-18a50691e936 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:21.477844 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f1f71-5b17-406c-8388-1482849bceef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:21.478438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f1f71-5b17-406c-8388-1482849bceef service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:21.478715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f1f71-5b17-406c-8388-1482849bceef service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:21.478987 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f1f71-5b17-406c-8388-1482849bceef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:21.487150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f1f71-5b17-406c-8388-1482849bceef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:21.488724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f1f71-5b17-406c-8388-1482849bceef service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:21.496842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1045/2074] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:31:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:31:25.034704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7942770-6867-4481-a3f0-d41a0e74ee99 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:25.035458 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7942770-6867-4481-a3f0-d41a0e74ee99 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:25.035458 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7942770-6867-4481-a3f0-d41a0e74ee99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:25.057079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c7942770-6867-4481-a3f0-d41a0e74ee99 None None] MFA Rules not processed for user `eca98759b55b443b9591eb6c80501157`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:25.058649 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c7942770-6867-4481-a3f0-d41a0e74ee99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:25.066273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1030/2075] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:31.493631 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:33.663257 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f1f71-5b17-406c-8388-1482849bceef service swift] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:33.730415 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f1f71-5b17-406c-8388-1482849bceef service swift] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:33.742306 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2d065-3aa8-480f-9eaa-d3d9e7aa384e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:33.743124 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2d065-3aa8-480f-9eaa-d3d9e7aa384e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:33.743516 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2d065-3aa8-480f-9eaa-d3d9e7aa384e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:33.743853 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2d065-3aa8-480f-9eaa-d3d9e7aa384e admin admin] PATH_INFO: `/v3/users/15b55f42d0554e11904da66af9645dbc` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:33.745547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2d065-3aa8-480f-9eaa-d3d9e7aa384e admin admin] RBAC: Authorizing `identity:delete_user(user_id=15b55f42d0554e11904da66af9645dbc)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:33.748338 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2d065-3aa8-480f-9eaa-d3d9e7aa384e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:33.850423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5ae2d065-3aa8-480f-9eaa-d3d9e7aa384e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15b55f42d0554e11904da66af9645dbc'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:33.863825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5ae2d065-3aa8-480f-9eaa-d3d9e7aa384e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15b55f42d0554e11904da66af9645dbc'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:33.906393 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5ae2d065-3aa8-480f-9eaa-d3d9e7aa384e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15b55f42d0554e11904da66af9645dbc'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:33.928340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5ae2d065-3aa8-480f-9eaa-d3d9e7aa384e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15b55f42d0554e11904da66af9645dbc'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:33.930303 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5ae2d065-3aa8-480f-9eaa-d3d9e7aa384e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15b55f42d0554e11904da66af9645dbc'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:33.946152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1046/2076] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:31:33 2022] DELETE /identity/v3/users/15b55f42d0554e11904da66af9645dbc => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:33.947604 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76907364-d5fd-4cc1-b3f6-ccc0e153fe28 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:33.948431 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76907364-d5fd-4cc1-b3f6-ccc0e153fe28 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:33.948864 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76907364-d5fd-4cc1-b3f6-ccc0e153fe28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:33.973248 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-76907364-d5fd-4cc1-b3f6-ccc0e153fe28 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:33.975683 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-76907364-d5fd-4cc1-b3f6-ccc0e153fe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:34.049763 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1031/2077] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:34.059957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2d065-3aa8-480f-9eaa-d3d9e7aa384e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:34.061392 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5ae2d065-3aa8-480f-9eaa-d3d9e7aa384e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:34.122757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2d065-3aa8-480f-9eaa-d3d9e7aa384e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:34.126457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2290f9-171f-499b-abdc-151905a87d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:34.127182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2290f9-171f-499b-abdc-151905a87d8b service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:34.127524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2290f9-171f-499b-abdc-151905a87d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:34.127861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2290f9-171f-499b-abdc-151905a87d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:34.140575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2290f9-171f-499b-abdc-151905a87d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:34.142119 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2290f9-171f-499b-abdc-151905a87d8b service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:34.159910 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1047/2078] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:31:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:31:34.505235 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76907364-d5fd-4cc1-b3f6-ccc0e153fe28 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:34.506976 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-76907364-d5fd-4cc1-b3f6-ccc0e153fe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:34.531664 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76907364-d5fd-4cc1-b3f6-ccc0e153fe28 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:34.554035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:34.556741 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:34.557214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:34.557659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] PATH_INFO: `/v3/projects/220eedf0de8b4b7dbaff769bd12af44a` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:34.560812 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=220eedf0de8b4b7dbaff769bd12af44a)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:34.565246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:34.661850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '220eedf0de8b4b7dbaff769bd12af44a', 'request_id': 'req-848d1c02-934b-4a22-8a2f-e3f4209f42f4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:34.668993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '220eedf0de8b4b7dbaff769bd12af44a', 'request_id': 'req-848d1c02-934b-4a22-8a2f-e3f4209f42f4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:34.675595 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '220eedf0de8b4b7dbaff769bd12af44a', 'request_id': 'req-848d1c02-934b-4a22-8a2f-e3f4209f42f4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:34.691032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] The token cache is being invalidate because project 220eedf0de8b4b7dbaff769bd12af44a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:31:34.692044 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:34.692832 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:34.697522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1032/2079] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:31:34 2022] DELETE /identity/v3/projects/220eedf0de8b4b7dbaff769bd12af44a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:34.698920 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2290f9-171f-499b-abdc-151905a87d8b service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:34.710603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2290f9-171f-499b-abdc-151905a87d8b service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:34.746946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:34.796947 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95649b74-a715-4b56-97f3-6dc0c4855e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:34.797691 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649b74-a715-4b56-97f3-6dc0c4855e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:34.797969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649b74-a715-4b56-97f3-6dc0c4855e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:34.798264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649b74-a715-4b56-97f3-6dc0c4855e2c admin admin] PATH_INFO: `/v3/users/aa84093f23f54eeca6770a3498340d5e` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:34.799423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95649b74-a715-4b56-97f3-6dc0c4855e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa84093f23f54eeca6770a3498340d5e)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:34.801935 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95649b74-a715-4b56-97f3-6dc0c4855e2c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:34.854919 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-95649b74-a715-4b56-97f3-6dc0c4855e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa84093f23f54eeca6770a3498340d5e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:34.863190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-95649b74-a715-4b56-97f3-6dc0c4855e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa84093f23f54eeca6770a3498340d5e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:34.876818 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-95649b74-a715-4b56-97f3-6dc0c4855e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa84093f23f54eeca6770a3498340d5e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:34.885087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-95649b74-a715-4b56-97f3-6dc0c4855e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa84093f23f54eeca6770a3498340d5e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:34.885465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-95649b74-a715-4b56-97f3-6dc0c4855e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa84093f23f54eeca6770a3498340d5e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:34.892258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1048/2080] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:31:34 2022] DELETE /identity/v3/users/aa84093f23f54eeca6770a3498340d5e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:35.394067 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:35.395669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:35.409716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d1c02-934b-4a22-8a2f-e3f4209f42f4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:35.486959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:35.487300 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:35.487581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:35.487856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a admin admin] PATH_INFO: `/v3/projects/487dc0aefb3645fba15de26d96ec0a7d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:35.489870 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=487dc0aefb3645fba15de26d96ec0a7d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:35.491902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:35.505668 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:35.565352 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '487dc0aefb3645fba15de26d96ec0a7d', 'request_id': 'req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:35.575282 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '487dc0aefb3645fba15de26d96ec0a7d', 'request_id': 'req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:35.584975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '487dc0aefb3645fba15de26d96ec0a7d', 'request_id': 'req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:35.598977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a admin admin] The token cache is being invalidate because project 487dc0aefb3645fba15de26d96ec0a7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:31:35.599303 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:35.600039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-edd0691c-6947-4ac5-ac5b-0da3e673bc2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:35.601930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1033/2081] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:31:35 2022] DELETE /identity/v3/projects/487dc0aefb3645fba15de26d96ec0a7d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:35.608016 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f2418-ad91-4094-a7b3-0730f9fcf314 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:35.608303 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f2418-ad91-4094-a7b3-0730f9fcf314 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:35.608582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f2418-ad91-4094-a7b3-0730f9fcf314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:35.644080 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-146f2418-ad91-4094-a7b3-0730f9fcf314 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:35.645916 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-146f2418-ad91-4094-a7b3-0730f9fcf314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:35.736518 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1049/2082] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:35.751542 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ca9d9-5c7b-4605-8ac3-617defd8cf82 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:35.751542 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ca9d9-5c7b-4605-8ac3-617defd8cf82 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:35.755134 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ca9d9-5c7b-4605-8ac3-617defd8cf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:35.803266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a24ca9d9-5c7b-4605-8ac3-617defd8cf82 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:35.804463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a24ca9d9-5c7b-4605-8ac3-617defd8cf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:35.829026 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1034/2083] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:35.834121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f2418-ad91-4094-a7b3-0730f9fcf314 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:35.844046 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f2418-ad91-4094-a7b3-0730f9fcf314 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:35.848887 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d8831-a589-4c96-a856-ba96082bfe3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:35.849499 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d8831-a589-4c96-a856-ba96082bfe3f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:35.849779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d8831-a589-4c96-a856-ba96082bfe3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:35.850053 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d8831-a589-4c96-a856-ba96082bfe3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:35.850737 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d8831-a589-4c96-a856-ba96082bfe3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:31:35.851047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d8831-a589-4c96-a856-ba96082bfe3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:35.852693 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d8831-a589-4c96-a856-ba96082bfe3f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:35.868185 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1050/2084] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:31:35.873783 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ca9d9-5c7b-4605-8ac3-617defd8cf82 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:35.884526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ca9d9-5c7b-4605-8ac3-617defd8cf82 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:35.889558 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df334d8-f9b8-45fc-b804-37555f519a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:35.890234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df334d8-f9b8-45fc-b804-37555f519a64 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:35.890629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df334d8-f9b8-45fc-b804-37555f519a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:35.890780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df334d8-f9b8-45fc-b804-37555f519a64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:35.892131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df334d8-f9b8-45fc-b804-37555f519a64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:35.893844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df334d8-f9b8-45fc-b804-37555f519a64 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:35.929092 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1035/2085] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:31:35 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:31:35.930248 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d8831-a589-4c96-a856-ba96082bfe3f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:35.942003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d8831-a589-4c96-a856-ba96082bfe3f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:35.970941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:36.008988 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edc302-183f-437f-80bb-03f484f71b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:36.009577 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edc302-183f-437f-80bb-03f484f71b6f admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:36.009887 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edc302-183f-437f-80bb-03f484f71b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:36.010115 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edc302-183f-437f-80bb-03f484f71b6f admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:36.010766 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edc302-183f-437f-80bb-03f484f71b6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:36.013024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edc302-183f-437f-80bb-03f484f71b6f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:36.036573 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1051/2086] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:31:35 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:31:36.041213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df334d8-f9b8-45fc-b804-37555f519a64 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:36.052444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df334d8-f9b8-45fc-b804-37555f519a64 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:36.057367 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9d46d-ae84-4033-acf7-342ab6792401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:36.057906 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9d46d-ae84-4033-acf7-342ab6792401 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:36.058186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9d46d-ae84-4033-acf7-342ab6792401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:36.058461 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9d46d-ae84-4033-acf7-342ab6792401 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:36.059132 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9d46d-ae84-4033-acf7-342ab6792401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:36.061122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9d46d-ae84-4033-acf7-342ab6792401 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:36.072865 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1036/2087] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:31:36.076937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edc302-183f-437f-80bb-03f484f71b6f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:36.088998 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edc302-183f-437f-80bb-03f484f71b6f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:36.094439 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88652c6-2e3b-4c49-84aa-a130d03bb3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:36.095041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88652c6-2e3b-4c49-84aa-a130d03bb3fd admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:36.095355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88652c6-2e3b-4c49-84aa-a130d03bb3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:36.095625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88652c6-2e3b-4c49-84aa-a130d03bb3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:36.096452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88652c6-2e3b-4c49-84aa-a130d03bb3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:36.098176 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88652c6-2e3b-4c49-84aa-a130d03bb3fd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:36.109390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1052/2088] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:31:36.114210 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9d46d-ae84-4033-acf7-342ab6792401 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:36.125424 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9d46d-ae84-4033-acf7-342ab6792401 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:36.130969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb9436-0c37-4484-975c-d23d057d26cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:36.131660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb9436-0c37-4484-975c-d23d057d26cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:36.131939 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb9436-0c37-4484-975c-d23d057d26cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:36.132209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb9436-0c37-4484-975c-d23d057d26cf admin admin] PATH_INFO: `/v3/projects/12a42dc28e8849ef80ed07ceced1a0e6/users/e574c26a1d98409b9a28bd26cf224554/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:36.148029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb9436-0c37-4484-975c-d23d057d26cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a42dc28e8849ef80ed07ceced1a0e6, user_id=e574c26a1d98409b9a28bd26cf224554, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:36.151261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb9436-0c37-4484-975c-d23d057d26cf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:36.165430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1037/2089] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:31:36 2022] PUT /identity/v3/projects/12a42dc28e8849ef80ed07ceced1a0e6/users/e574c26a1d98409b9a28bd26cf224554/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:36.177318 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65868ea6-a5ca-4e1c-9b48-1d749de19f77 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:36.177604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65868ea6-a5ca-4e1c-9b48-1d749de19f77 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:36.177882 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65868ea6-a5ca-4e1c-9b48-1d749de19f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:36.207844 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-65868ea6-a5ca-4e1c-9b48-1d749de19f77 None None] MFA Rules not processed for user `e574c26a1d98409b9a28bd26cf224554`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:36.209071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-65868ea6-a5ca-4e1c-9b48-1d749de19f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:36.279390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1053/2090] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:36.284309 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb9436-0c37-4484-975c-d23d057d26cf admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:36.351793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb9436-0c37-4484-975c-d23d057d26cf admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:36.357303 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b7856-0b37-4143-9e5a-3aa512cb8f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:36.358231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b7856-0b37-4143-9e5a-3aa512cb8f2d admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:36.358231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b7856-0b37-4143-9e5a-3aa512cb8f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:36.358231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b7856-0b37-4143-9e5a-3aa512cb8f2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:36.358887 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b7856-0b37-4143-9e5a-3aa512cb8f2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:36.361013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b7856-0b37-4143-9e5a-3aa512cb8f2d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:36.407275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1038/2091] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:31:36 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:31:36.411903 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65868ea6-a5ca-4e1c-9b48-1d749de19f77 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:36.426717 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65868ea6-a5ca-4e1c-9b48-1d749de19f77 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:36.510073 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa39d2-650b-47c1-9591-f21ff4128fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:36.511320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa39d2-650b-47c1-9591-f21ff4128fcc admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:36.511608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa39d2-650b-47c1-9591-f21ff4128fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:36.511886 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa39d2-650b-47c1-9591-f21ff4128fcc admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:36.512535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa39d2-650b-47c1-9591-f21ff4128fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:36.515330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa39d2-650b-47c1-9591-f21ff4128fcc admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:36.540597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1054/2092] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:31:36 2022] POST /identity/v3/users => generated 296 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:31:36.544769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b7856-0b37-4143-9e5a-3aa512cb8f2d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:36.558333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b7856-0b37-4143-9e5a-3aa512cb8f2d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:36.564259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac8fad-f85b-444c-a3ef-e568cd9cb961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:36.565105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac8fad-f85b-444c-a3ef-e568cd9cb961 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:36.565394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac8fad-f85b-444c-a3ef-e568cd9cb961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:36.566158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac8fad-f85b-444c-a3ef-e568cd9cb961 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:36.567266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac8fad-f85b-444c-a3ef-e568cd9cb961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:36.569590 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac8fad-f85b-444c-a3ef-e568cd9cb961 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:36.588991 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1039/2093] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:31:36.593586 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa39d2-650b-47c1-9591-f21ff4128fcc admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:36.617325 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa39d2-650b-47c1-9591-f21ff4128fcc admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:36.626858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fb88e-4e52-4240-9a85-fbdeed724433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:36.627575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fb88e-4e52-4240-9a85-fbdeed724433 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:36.629864 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fb88e-4e52-4240-9a85-fbdeed724433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:36.630572 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fb88e-4e52-4240-9a85-fbdeed724433 admin admin] PATH_INFO: `/v3/projects/d9fac6372463417d8e33161c591a1cef/users/1e899cc923bd45a482093c812e72f1da/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:36.652922 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fb88e-4e52-4240-9a85-fbdeed724433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9fac6372463417d8e33161c591a1cef, user_id=1e899cc923bd45a482093c812e72f1da, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:36.657680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937fb88e-4e52-4240-9a85-fbdeed724433 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:36.677128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1055/2094] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:31:36 2022] PUT /identity/v3/projects/d9fac6372463417d8e33161c591a1cef/users/1e899cc923bd45a482093c812e72f1da/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:36.681383 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac8fad-f85b-444c-a3ef-e568cd9cb961 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:36.692752 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac8fad-f85b-444c-a3ef-e568cd9cb961 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:36.704884 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:36.768791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99361660-9c18-473d-99b1-dc7e4cf54384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:36.769626 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99361660-9c18-473d-99b1-dc7e4cf54384 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:36.769836 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99361660-9c18-473d-99b1-dc7e4cf54384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:36.770198 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99361660-9c18-473d-99b1-dc7e4cf54384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:36.770923 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99361660-9c18-473d-99b1-dc7e4cf54384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:36.772686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99361660-9c18-473d-99b1-dc7e4cf54384 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:36.784014 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1040/2095] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:31:36.788191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fb88e-4e52-4240-9a85-fbdeed724433 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:36.793861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:36.800201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fb88e-4e52-4240-9a85-fbdeed724433 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:36.808755 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13daf6ff-febd-4aaf-a521-5b80e17c4deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:36.809688 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daf6ff-febd-4aaf-a521-5b80e17c4deb admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:36.810138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daf6ff-febd-4aaf-a521-5b80e17c4deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:36.810564 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daf6ff-febd-4aaf-a521-5b80e17c4deb admin admin] PATH_INFO: `/v3/projects/d9fac6372463417d8e33161c591a1cef/users/1e899cc923bd45a482093c812e72f1da/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:36.812982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13daf6ff-febd-4aaf-a521-5b80e17c4deb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9fac6372463417d8e33161c591a1cef, user_id=1e899cc923bd45a482093c812e72f1da, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:36.817507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13daf6ff-febd-4aaf-a521-5b80e17c4deb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:36.830219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1056/2096] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:31:36 2022] PUT /identity/v3/projects/d9fac6372463417d8e33161c591a1cef/users/1e899cc923bd45a482093c812e72f1da/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:36.841838 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585be4c-191c-4ddf-9a82-6444f0091b0c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:36.842125 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585be4c-191c-4ddf-9a82-6444f0091b0c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:36.842404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585be4c-191c-4ddf-9a82-6444f0091b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:36.876428 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f585be4c-191c-4ddf-9a82-6444f0091b0c None None] MFA Rules not processed for user `1e899cc923bd45a482093c812e72f1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:36.877666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f585be4c-191c-4ddf-9a82-6444f0091b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:36.958149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1041/2097] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:36.971355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13daf6ff-febd-4aaf-a521-5b80e17c4deb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:36.973645 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-13daf6ff-febd-4aaf-a521-5b80e17c4deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:37.039147 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13daf6ff-febd-4aaf-a521-5b80e17c4deb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:37.042773 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9139d8b-5aa3-4aff-b5ed-aa675b6476ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:37.043418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9139d8b-5aa3-4aff-b5ed-aa675b6476ac service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:37.043699 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9139d8b-5aa3-4aff-b5ed-aa675b6476ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:37.043999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9139d8b-5aa3-4aff-b5ed-aa675b6476ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:37.053451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9139d8b-5aa3-4aff-b5ed-aa675b6476ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:37.054887 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9139d8b-5aa3-4aff-b5ed-aa675b6476ac service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:37.064209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1057/2098] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:31:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:31:37.370503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585be4c-191c-4ddf-9a82-6444f0091b0c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:37.380268 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585be4c-191c-4ddf-9a82-6444f0091b0c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:37.384869 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3769658-c856-415a-9fe1-b4829489273b tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e899cc923bd45a482093c812e72f1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9fac6372463417d8e33161c591a1cef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:37.385482 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3769658-c856-415a-9fe1-b4829489273b tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:37.385765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3769658-c856-415a-9fe1-b4829489273b tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:37.386189 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3769658-c856-415a-9fe1-b4829489273b tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:37.386837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3769658-c856-415a-9fe1-b4829489273b tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:31:37.387168 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3769658-c856-415a-9fe1-b4829489273b tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:37.388880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3769658-c856-415a-9fe1-b4829489273b tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:37.401306 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1042/2099] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:31:37.405555 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9139d8b-5aa3-4aff-b5ed-aa675b6476ac service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:37.415390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9139d8b-5aa3-4aff-b5ed-aa675b6476ac service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:37.420000 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33427b26-c177-4398-acd4-7d8917dace64 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e899cc923bd45a482093c812e72f1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9fac6372463417d8e33161c591a1cef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:37.420687 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427b26-c177-4398-acd4-7d8917dace64 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:37.421036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427b26-c177-4398-acd4-7d8917dace64 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:37.421378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427b26-c177-4398-acd4-7d8917dace64 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:37.422072 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33427b26-c177-4398-acd4-7d8917dace64 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:37.424104 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33427b26-c177-4398-acd4-7d8917dace64 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:37.453037 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1058/2100] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:31:37 2022] POST /identity/v3/projects => generated 353 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:31:37.457452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3769658-c856-415a-9fe1-b4829489273b tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:37.468849 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3769658-c856-415a-9fe1-b4829489273b tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:37.530708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36670b4e-3a17-43fa-86f8-c0ef45c3555e tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e899cc923bd45a482093c812e72f1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9fac6372463417d8e33161c591a1cef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:37.531602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36670b4e-3a17-43fa-86f8-c0ef45c3555e tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:37.531938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36670b4e-3a17-43fa-86f8-c0ef45c3555e tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:37.532338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36670b4e-3a17-43fa-86f8-c0ef45c3555e tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:37.533023 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36670b4e-3a17-43fa-86f8-c0ef45c3555e tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:31:37.533304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36670b4e-3a17-43fa-86f8-c0ef45c3555e tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:37.535305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36670b4e-3a17-43fa-86f8-c0ef45c3555e tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:37.552948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1043/2101] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:31:37.589451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33427b26-c177-4398-acd4-7d8917dace64 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:37.616005 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33427b26-c177-4398-acd4-7d8917dace64 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:37.620827 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e899cc923bd45a482093c812e72f1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9fac6372463417d8e33161c591a1cef', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:37.621471 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:37.621750 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:37.622020 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] PATH_INFO: `/v3/projects/48e509b06ab745a4b7968ceb2d87d3a7` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:37.623188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] RBAC: Authorizing `identity:delete_project(project_id=48e509b06ab745a4b7968ceb2d87d3a7)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:37.625100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:37.673788 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48e509b06ab745a4b7968ceb2d87d3a7', 'request_id': 'req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:37.679398 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48e509b06ab745a4b7968ceb2d87d3a7', 'request_id': 'req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:37.686122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48e509b06ab745a4b7968ceb2d87d3a7', 'request_id': 'req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:37.698178 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] The token cache is being invalidate because project 48e509b06ab745a4b7968ceb2d87d3a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:31:37.698484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:37.699124 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6ffc7d96-6dd8-4ba2-b0f9-243134d88e88 tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:37.700623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1059/2102] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:31:37 2022] DELETE /identity/v3/projects/48e509b06ab745a4b7968ceb2d87d3a7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:37.896938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36670b4e-3a17-43fa-86f8-c0ef45c3555e tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:37.898864 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-36670b4e-3a17-43fa-86f8-c0ef45c3555e tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:37.914047 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:37.966122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36670b4e-3a17-43fa-86f8-c0ef45c3555e tempest-VolumeQuotaClassesTest-188702631 tempest-VolumeQuotaClassesTest-188702631-project-admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:38.023201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb9e6f-0712-4128-91c8-b08313393b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:38.023848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb9e6f-0712-4128-91c8-b08313393b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:38.024122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb9e6f-0712-4128-91c8-b08313393b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:38.024396 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb9e6f-0712-4128-91c8-b08313393b23 admin admin] PATH_INFO: `/v3/users/e574c26a1d98409b9a28bd26cf224554` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:38.025536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb9e6f-0712-4128-91c8-b08313393b23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e574c26a1d98409b9a28bd26cf224554)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:38.027490 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb9e6f-0712-4128-91c8-b08313393b23 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:38.082884 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0efb9e6f-0712-4128-91c8-b08313393b23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e574c26a1d98409b9a28bd26cf224554'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.093309 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0efb9e6f-0712-4128-91c8-b08313393b23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e574c26a1d98409b9a28bd26cf224554'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.094004 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0efb9e6f-0712-4128-91c8-b08313393b23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e574c26a1d98409b9a28bd26cf224554'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.107474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0efb9e6f-0712-4128-91c8-b08313393b23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e574c26a1d98409b9a28bd26cf224554'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.115561 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0efb9e6f-0712-4128-91c8-b08313393b23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e574c26a1d98409b9a28bd26cf224554'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.121781 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1044/2103] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:31:37 2022] DELETE /identity/v3/users/e574c26a1d98409b9a28bd26cf224554 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:31:38.126141 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df147a7-090a-4c69-ba0b-56d1f650f044 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:38.126595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df147a7-090a-4c69-ba0b-56d1f650f044 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:38.126704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df147a7-090a-4c69-ba0b-56d1f650f044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:38.146377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6df147a7-090a-4c69-ba0b-56d1f650f044 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:38.147670 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6df147a7-090a-4c69-ba0b-56d1f650f044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:38.197312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:38.210930 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1060/2104] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:38.220829 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb9e6f-0712-4128-91c8-b08313393b23 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:38.222214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0efb9e6f-0712-4128-91c8-b08313393b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:38.265893 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:38.281572 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb9e6f-0712-4128-91c8-b08313393b23 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:38.285320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa65da6-b42b-4a2a-bff5-ce0c23e7bd7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:38.285923 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa65da6-b42b-4a2a-bff5-ce0c23e7bd7d service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:38.286199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa65da6-b42b-4a2a-bff5-ce0c23e7bd7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:38.286468 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa65da6-b42b-4a2a-bff5-ce0c23e7bd7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:38.295233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa65da6-b42b-4a2a-bff5-ce0c23e7bd7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:38.296641 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa65da6-b42b-4a2a-bff5-ce0c23e7bd7d service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:38.306369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1045/2105] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:31:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:31:38.569613 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df147a7-090a-4c69-ba0b-56d1f650f044 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:38.571032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6df147a7-090a-4c69-ba0b-56d1f650f044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:38.584261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df147a7-090a-4c69-ba0b-56d1f650f044 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:38.588837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a86cec-da06-4d17-b611-3aeb64da5963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:38.589468 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a86cec-da06-4d17-b611-3aeb64da5963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:38.589746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a86cec-da06-4d17-b611-3aeb64da5963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:38.590017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a86cec-da06-4d17-b611-3aeb64da5963 admin admin] PATH_INFO: `/v3/projects/12a42dc28e8849ef80ed07ceced1a0e6` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:38.591160 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a86cec-da06-4d17-b611-3aeb64da5963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12a42dc28e8849ef80ed07ceced1a0e6)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:38.593099 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a86cec-da06-4d17-b611-3aeb64da5963 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:38.645605 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-74a86cec-da06-4d17-b611-3aeb64da5963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12a42dc28e8849ef80ed07ceced1a0e6', 'request_id': 'req-74a86cec-da06-4d17-b611-3aeb64da5963'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.651729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-74a86cec-da06-4d17-b611-3aeb64da5963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12a42dc28e8849ef80ed07ceced1a0e6', 'request_id': 'req-74a86cec-da06-4d17-b611-3aeb64da5963'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.658276 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-74a86cec-da06-4d17-b611-3aeb64da5963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12a42dc28e8849ef80ed07ceced1a0e6', 'request_id': 'req-74a86cec-da06-4d17-b611-3aeb64da5963'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.678757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-74a86cec-da06-4d17-b611-3aeb64da5963 admin admin] The token cache is being invalidate because project 12a42dc28e8849ef80ed07ceced1a0e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:31:38.679056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-74a86cec-da06-4d17-b611-3aeb64da5963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.679703 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-74a86cec-da06-4d17-b611-3aeb64da5963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.681442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1061/2106] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:31:38 2022] DELETE /identity/v3/projects/12a42dc28e8849ef80ed07ceced1a0e6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:38.685090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa65da6-b42b-4a2a-bff5-ce0c23e7bd7d service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:38.694044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27226f79-baec-40c5-af97-d362bad3f46b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:38.694372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27226f79-baec-40c5-af97-d362bad3f46b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:38.694657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27226f79-baec-40c5-af97-d362bad3f46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:38.696178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa65da6-b42b-4a2a-bff5-ce0c23e7bd7d service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:38.717628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-27226f79-baec-40c5-af97-d362bad3f46b None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:38.718833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-27226f79-baec-40c5-af97-d362bad3f46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:38.761656 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5cb64-8f07-4ed3-999f-a0980944b7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:38.762543 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5cb64-8f07-4ed3-999f-a0980944b7f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:38.763404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5cb64-8f07-4ed3-999f-a0980944b7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:38.764668 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5cb64-8f07-4ed3-999f-a0980944b7f1 admin admin] PATH_INFO: `/v3/users/1e899cc923bd45a482093c812e72f1da` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:38.766030 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5cb64-8f07-4ed3-999f-a0980944b7f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e899cc923bd45a482093c812e72f1da)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:38.768320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5cb64-8f07-4ed3-999f-a0980944b7f1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:38.790495 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1062/2107] 10.222.0.18 () {58 vars in 967 bytes} [Fri Oct 14 02:31:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:38.818835 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27226f79-baec-40c5-af97-d362bad3f46b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:38.820963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-27226f79-baec-40c5-af97-d362bad3f46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:38.823822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a2e5cb64-8f07-4ed3-999f-a0980944b7f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e899cc923bd45a482093c812e72f1da'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.833840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a2e5cb64-8f07-4ed3-999f-a0980944b7f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e899cc923bd45a482093c812e72f1da'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.834727 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a2e5cb64-8f07-4ed3-999f-a0980944b7f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e899cc923bd45a482093c812e72f1da'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.850928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a2e5cb64-8f07-4ed3-999f-a0980944b7f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e899cc923bd45a482093c812e72f1da'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.859978 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a2e5cb64-8f07-4ed3-999f-a0980944b7f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e899cc923bd45a482093c812e72f1da'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:38.867672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1046/2108] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:31:38 2022] DELETE /identity/v3/users/1e899cc923bd45a482093c812e72f1da => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:38.905005 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27226f79-baec-40c5-af97-d362bad3f46b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:38.933863 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:38.984481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e9000-2073-4027-b4ac-34de86f61187 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:38.985245 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e9000-2073-4027-b4ac-34de86f61187 service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:38.986134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e9000-2073-4027-b4ac-34de86f61187 service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:38.986134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e9000-2073-4027-b4ac-34de86f61187 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:38.989098 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8f5e9000-2073-4027-b4ac-34de86f61187 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:39.061672 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e9000-2073-4027-b4ac-34de86f61187 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:39.063525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e9000-2073-4027-b4ac-34de86f61187 service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:39.073713 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1063/2109] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:31:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 14 02:31:39.195123 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5cb64-8f07-4ed3-999f-a0980944b7f1 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:39.196571 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a2e5cb64-8f07-4ed3-999f-a0980944b7f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:39.284793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5cb64-8f07-4ed3-999f-a0980944b7f1 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:39.296059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ef272-dde3-405c-917f-7dd64baa2e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:39.296972 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ef272-dde3-405c-917f-7dd64baa2e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:39.297342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ef272-dde3-405c-917f-7dd64baa2e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:39.297752 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ef272-dde3-405c-917f-7dd64baa2e77 admin admin] PATH_INFO: `/v3/projects/d9fac6372463417d8e33161c591a1cef` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:39.299516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ef272-dde3-405c-917f-7dd64baa2e77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9fac6372463417d8e33161c591a1cef)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:39.302284 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ef272-dde3-405c-917f-7dd64baa2e77 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:39.304714 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:39.360534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b65ef272-dde3-405c-917f-7dd64baa2e77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9fac6372463417d8e33161c591a1cef', 'request_id': 'req-b65ef272-dde3-405c-917f-7dd64baa2e77'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:39.366868 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b65ef272-dde3-405c-917f-7dd64baa2e77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9fac6372463417d8e33161c591a1cef', 'request_id': 'req-b65ef272-dde3-405c-917f-7dd64baa2e77'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:39.374224 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b65ef272-dde3-405c-917f-7dd64baa2e77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9fac6372463417d8e33161c591a1cef', 'request_id': 'req-b65ef272-dde3-405c-917f-7dd64baa2e77'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:39.390166 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b65ef272-dde3-405c-917f-7dd64baa2e77 admin admin] The token cache is being invalidate because project d9fac6372463417d8e33161c591a1cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:31:39.390455 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b65ef272-dde3-405c-917f-7dd64baa2e77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:39.391158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b65ef272-dde3-405c-917f-7dd64baa2e77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:39.396819 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1047/2110] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:31:39 2022] DELETE /identity/v3/projects/d9fac6372463417d8e33161c591a1cef => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:39.398523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9364e-9b0f-465d-a4b1-411ed6021059 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:39.399012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9364e-9b0f-465d-a4b1-411ed6021059 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:39.399192 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9364e-9b0f-465d-a4b1-411ed6021059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:39.423055 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a7b9364e-9b0f-465d-a4b1-411ed6021059 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:39.424557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a7b9364e-9b0f-465d-a4b1-411ed6021059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:39.500787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1064/2111] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:39.509629 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad4000-e774-4826-a945-e92c8e7b34e6 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:39.509912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad4000-e774-4826-a945-e92c8e7b34e6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:39.510186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad4000-e774-4826-a945-e92c8e7b34e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:39.531743 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e5ad4000-e774-4826-a945-e92c8e7b34e6 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:39.532961 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e5ad4000-e774-4826-a945-e92c8e7b34e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:39.541184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1048/2112] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:39.545345 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9364e-9b0f-465d-a4b1-411ed6021059 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:39.555743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9364e-9b0f-465d-a4b1-411ed6021059 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:39.561593 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01482d-fd8b-455f-afcd-7e5453900c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:39.562392 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01482d-fd8b-455f-afcd-7e5453900c22 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:39.562837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01482d-fd8b-455f-afcd-7e5453900c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:39.563170 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01482d-fd8b-455f-afcd-7e5453900c22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:39.563817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01482d-fd8b-455f-afcd-7e5453900c22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:31:39.564125 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01482d-fd8b-455f-afcd-7e5453900c22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:39.565949 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01482d-fd8b-455f-afcd-7e5453900c22 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:39.578594 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1065/2113] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:31:39.583914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad4000-e774-4826-a945-e92c8e7b34e6 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:39.593045 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad4000-e774-4826-a945-e92c8e7b34e6 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:39.597750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7789a-4205-4bc8-b42e-d9d5360f2df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:39.598338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7789a-4205-4bc8-b42e-d9d5360f2df0 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:39.598657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7789a-4205-4bc8-b42e-d9d5360f2df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:39.598925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7789a-4205-4bc8-b42e-d9d5360f2df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:39.599581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7789a-4205-4bc8-b42e-d9d5360f2df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:39.601474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7789a-4205-4bc8-b42e-d9d5360f2df0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:39.629946 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1049/2114] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:31:39 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:31:39.633374 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01482d-fd8b-455f-afcd-7e5453900c22 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:39.643491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01482d-fd8b-455f-afcd-7e5453900c22 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:39.694498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:39.705364 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af7e50-5bda-448d-8cb3-f4392d91934e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:39.706060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af7e50-5bda-448d-8cb3-f4392d91934e admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:39.706364 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af7e50-5bda-448d-8cb3-f4392d91934e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:39.706672 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af7e50-5bda-448d-8cb3-f4392d91934e admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:39.707465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af7e50-5bda-448d-8cb3-f4392d91934e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:39.710002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af7e50-5bda-448d-8cb3-f4392d91934e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:39.734224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1066/2115] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:31:39 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:31:39.739455 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7789a-4205-4bc8-b42e-d9d5360f2df0 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:39.751155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7789a-4205-4bc8-b42e-d9d5360f2df0 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:39.756505 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9ead2-470d-428a-969c-ced1b5f91e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:39.757117 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9ead2-470d-428a-969c-ced1b5f91e25 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:39.757394 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9ead2-470d-428a-969c-ced1b5f91e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:39.757726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9ead2-470d-428a-969c-ced1b5f91e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:39.758563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9ead2-470d-428a-969c-ced1b5f91e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:39.760272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9ead2-470d-428a-969c-ced1b5f91e25 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:39.771053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1050/2116] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:31:39.775030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af7e50-5bda-448d-8cb3-f4392d91934e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:39.785306 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af7e50-5bda-448d-8cb3-f4392d91934e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:39.789900 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791310ad-f618-4638-a52a-a1db064bcbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:39.790510 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791310ad-f618-4638-a52a-a1db064bcbf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:39.790794 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791310ad-f618-4638-a52a-a1db064bcbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:39.791084 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791310ad-f618-4638-a52a-a1db064bcbf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:39.791780 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791310ad-f618-4638-a52a-a1db064bcbf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:39.793563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791310ad-f618-4638-a52a-a1db064bcbf3 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:39.804849 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1067/2117] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:31:39.808896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9ead2-470d-428a-969c-ced1b5f91e25 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:39.819578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9ead2-470d-428a-969c-ced1b5f91e25 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:39.824076 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65720cec-21d6-499e-b941-d00309c54729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:39.824742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65720cec-21d6-499e-b941-d00309c54729 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:39.825020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65720cec-21d6-499e-b941-d00309c54729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:39.825290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65720cec-21d6-499e-b941-d00309c54729 admin admin] PATH_INFO: `/v3/projects/c3879c607feb4ff3a757dda8bb270dc5/users/f644b2b3468e473faa155b402c54d7eb/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:39.839398 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65720cec-21d6-499e-b941-d00309c54729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3879c607feb4ff3a757dda8bb270dc5, user_id=f644b2b3468e473faa155b402c54d7eb, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:39.842073 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65720cec-21d6-499e-b941-d00309c54729 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:39.854296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1051/2118] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:31:39 2022] PUT /identity/v3/projects/c3879c607feb4ff3a757dda8bb270dc5/users/f644b2b3468e473faa155b402c54d7eb/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:39.865732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66045b36-b8f7-4364-92da-7415f8f2564c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:39.866017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66045b36-b8f7-4364-92da-7415f8f2564c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:39.866323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66045b36-b8f7-4364-92da-7415f8f2564c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:39.894444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-66045b36-b8f7-4364-92da-7415f8f2564c None None] MFA Rules not processed for user `f644b2b3468e473faa155b402c54d7eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:39.895642 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-66045b36-b8f7-4364-92da-7415f8f2564c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:39.958530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1068/2119] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:39.962903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65720cec-21d6-499e-b941-d00309c54729 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:40.023604 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65720cec-21d6-499e-b941-d00309c54729 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:40.028190 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8821ddb-20f6-4aa5-a3b0-ff5da5ad3741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:40.028798 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8821ddb-20f6-4aa5-a3b0-ff5da5ad3741 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:40.029071 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8821ddb-20f6-4aa5-a3b0-ff5da5ad3741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:40.029342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8821ddb-20f6-4aa5-a3b0-ff5da5ad3741 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:40.030185 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8821ddb-20f6-4aa5-a3b0-ff5da5ad3741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:40.032396 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8821ddb-20f6-4aa5-a3b0-ff5da5ad3741 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:40.061571 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1052/2120] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:31:39 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:31:40.064418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66045b36-b8f7-4364-92da-7415f8f2564c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:40.074817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66045b36-b8f7-4364-92da-7415f8f2564c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:40.132385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e740776-a187-4c90-a5fa-57cf5dcf22e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:40.132998 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e740776-a187-4c90-a5fa-57cf5dcf22e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:40.133276 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e740776-a187-4c90-a5fa-57cf5dcf22e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:40.133544 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e740776-a187-4c90-a5fa-57cf5dcf22e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:40.134181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e740776-a187-4c90-a5fa-57cf5dcf22e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:40.136137 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e740776-a187-4c90-a5fa-57cf5dcf22e3 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:40.156967 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1069/2121] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:31:40 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:31:40.161638 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8821ddb-20f6-4aa5-a3b0-ff5da5ad3741 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:40.172073 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8821ddb-20f6-4aa5-a3b0-ff5da5ad3741 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:40.177350 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b69a8-2064-4c95-87f1-28068b30ed6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:40.177955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b69a8-2064-4c95-87f1-28068b30ed6d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:40.178238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b69a8-2064-4c95-87f1-28068b30ed6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:40.178508 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b69a8-2064-4c95-87f1-28068b30ed6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:40.179182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b69a8-2064-4c95-87f1-28068b30ed6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:40.180794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b69a8-2064-4c95-87f1-28068b30ed6d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:40.192535 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1053/2122] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:31:40.196511 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e740776-a187-4c90-a5fa-57cf5dcf22e3 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:40.206256 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e740776-a187-4c90-a5fa-57cf5dcf22e3 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:40.210931 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93242cc-ce88-4436-ae5c-36ce98466e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:40.211510 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93242cc-ce88-4436-ae5c-36ce98466e13 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:40.211786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93242cc-ce88-4436-ae5c-36ce98466e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:40.212058 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93242cc-ce88-4436-ae5c-36ce98466e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:40.212690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93242cc-ce88-4436-ae5c-36ce98466e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:40.214283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93242cc-ce88-4436-ae5c-36ce98466e13 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:40.224608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1070/2123] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:31:40.228534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b69a8-2064-4c95-87f1-28068b30ed6d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:40.233950 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:40.241125 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b69a8-2064-4c95-87f1-28068b30ed6d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:40.246641 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad467a5-f7f8-4b90-939e-ea9035f00b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:40.247378 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad467a5-f7f8-4b90-939e-ea9035f00b67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:40.247698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad467a5-f7f8-4b90-939e-ea9035f00b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:40.248017 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad467a5-f7f8-4b90-939e-ea9035f00b67 admin admin] PATH_INFO: `/v3/projects/eb236344569d45ccbd510a05586de3df/users/2c28e0a7d7174e8ea0679f00fbf91dbf/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:40.263096 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad467a5-f7f8-4b90-939e-ea9035f00b67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb236344569d45ccbd510a05586de3df, user_id=2c28e0a7d7174e8ea0679f00fbf91dbf, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:40.265908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad467a5-f7f8-4b90-939e-ea9035f00b67 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:40.279156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1054/2124] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:31:40 2022] PUT /identity/v3/projects/eb236344569d45ccbd510a05586de3df/users/2c28e0a7d7174e8ea0679f00fbf91dbf/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:40.290943 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c801e-976b-4e21-8e57-b11cfe094ef5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:40.291246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c801e-976b-4e21-8e57-b11cfe094ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:40.291520 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c801e-976b-4e21-8e57-b11cfe094ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:40.319461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-816c801e-976b-4e21-8e57-b11cfe094ef5 None None] MFA Rules not processed for user `2c28e0a7d7174e8ea0679f00fbf91dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:40.320663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-816c801e-976b-4e21-8e57-b11cfe094ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:40.349908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:40.384921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1071/2125] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:40.389370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad467a5-f7f8-4b90-939e-ea9035f00b67 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:40.449691 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad467a5-f7f8-4b90-939e-ea9035f00b67 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:40.454334 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c58ad-7af9-47c7-8416-2c004cf8af70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:40.454937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c58ad-7af9-47c7-8416-2c004cf8af70 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:40.455262 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c58ad-7af9-47c7-8416-2c004cf8af70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:40.455534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c58ad-7af9-47c7-8416-2c004cf8af70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:40.456168 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c58ad-7af9-47c7-8416-2c004cf8af70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:40.458592 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c58ad-7af9-47c7-8416-2c004cf8af70 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:40.484841 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1055/2126] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:31:40 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:31:40.489144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c801e-976b-4e21-8e57-b11cfe094ef5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:40.500052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c801e-976b-4e21-8e57-b11cfe094ef5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:40.566532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45b6b6-139c-4096-85c2-3b222ef24f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:40.567497 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45b6b6-139c-4096-85c2-3b222ef24f80 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:40.567936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45b6b6-139c-4096-85c2-3b222ef24f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:40.568385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45b6b6-139c-4096-85c2-3b222ef24f80 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:40.569194 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45b6b6-139c-4096-85c2-3b222ef24f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:40.571305 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45b6b6-139c-4096-85c2-3b222ef24f80 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:40.595103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1072/2127] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:31:40 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:31:40.599853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c58ad-7af9-47c7-8416-2c004cf8af70 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:40.610686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c58ad-7af9-47c7-8416-2c004cf8af70 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:40.615356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3d3e6-50f9-4c23-a985-a759cd211e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:40.615965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3d3e6-50f9-4c23-a985-a759cd211e2a admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:40.616254 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3d3e6-50f9-4c23-a985-a759cd211e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:40.616516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3d3e6-50f9-4c23-a985-a759cd211e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:40.617157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3d3e6-50f9-4c23-a985-a759cd211e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:40.619425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3d3e6-50f9-4c23-a985-a759cd211e2a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:40.632366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1056/2128] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:31:40.636588 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45b6b6-139c-4096-85c2-3b222ef24f80 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:40.648198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45b6b6-139c-4096-85c2-3b222ef24f80 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:40.653108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878644cb-024f-4798-a5fd-fdd7aa7206c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:40.653931 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878644cb-024f-4798-a5fd-fdd7aa7206c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:40.654279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878644cb-024f-4798-a5fd-fdd7aa7206c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:40.654625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878644cb-024f-4798-a5fd-fdd7aa7206c1 admin admin] PATH_INFO: `/v3/projects/b33fa8ce9aa54b889d66e57de508821d/users/1fe30007bf8747d68ad619c000661fb0/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:40.671198 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878644cb-024f-4798-a5fd-fdd7aa7206c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33fa8ce9aa54b889d66e57de508821d, user_id=1fe30007bf8747d68ad619c000661fb0, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:40.673877 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878644cb-024f-4798-a5fd-fdd7aa7206c1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:40.687504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1073/2129] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:31:40 2022] PUT /identity/v3/projects/b33fa8ce9aa54b889d66e57de508821d/users/1fe30007bf8747d68ad619c000661fb0/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:31:40.692462 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3d3e6-50f9-4c23-a985-a759cd211e2a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:40.702981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3d3e6-50f9-4c23-a985-a759cd211e2a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:40.775924 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913baa2e-1ca4-40d7-afed-30657736fdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:40.776569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913baa2e-1ca4-40d7-afed-30657736fdb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:40.776800 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913baa2e-1ca4-40d7-afed-30657736fdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:40.777067 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913baa2e-1ca4-40d7-afed-30657736fdb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:40.777708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913baa2e-1ca4-40d7-afed-30657736fdb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:40.779316 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913baa2e-1ca4-40d7-afed-30657736fdb4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:40.789513 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1057/2130] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:31:40.793593 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878644cb-024f-4798-a5fd-fdd7aa7206c1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:40.804543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878644cb-024f-4798-a5fd-fdd7aa7206c1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:40.809081 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e21dc-82e8-41da-b12c-4433c421fdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:40.809729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e21dc-82e8-41da-b12c-4433c421fdef admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:40.810008 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e21dc-82e8-41da-b12c-4433c421fdef admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:40.810280 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e21dc-82e8-41da-b12c-4433c421fdef admin admin] PATH_INFO: `/v3/projects/b33fa8ce9aa54b889d66e57de508821d/users/1fe30007bf8747d68ad619c000661fb0/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:40.812154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e21dc-82e8-41da-b12c-4433c421fdef admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33fa8ce9aa54b889d66e57de508821d, user_id=1fe30007bf8747d68ad619c000661fb0, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:40.814773 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e21dc-82e8-41da-b12c-4433c421fdef admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:40.825156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1074/2131] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:31:40 2022] PUT /identity/v3/projects/b33fa8ce9aa54b889d66e57de508821d/users/1fe30007bf8747d68ad619c000661fb0/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:40.835034 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d121bb-cdbd-44bc-9efc-9764e900115f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:40.835393 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d121bb-cdbd-44bc-9efc-9764e900115f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:40.835669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d121bb-cdbd-44bc-9efc-9764e900115f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:40.864090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-04d121bb-cdbd-44bc-9efc-9764e900115f None None] MFA Rules not processed for user `1fe30007bf8747d68ad619c000661fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:40.865306 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-04d121bb-cdbd-44bc-9efc-9764e900115f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:40.901474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:40.933056 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1058/2132] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:40.944714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e21dc-82e8-41da-b12c-4433c421fdef admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:40.946416 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-629e21dc-82e8-41da-b12c-4433c421fdef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:41.004156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e21dc-82e8-41da-b12c-4433c421fdef admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:41.007806 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e71d1-9379-4e4e-9497-ff48c8834919 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:41.008404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e71d1-9379-4e4e-9497-ff48c8834919 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:41.008686 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e71d1-9379-4e4e-9497-ff48c8834919 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:41.008956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e71d1-9379-4e4e-9497-ff48c8834919 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:41.017705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e71d1-9379-4e4e-9497-ff48c8834919 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:41.019550 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e71d1-9379-4e4e-9497-ff48c8834919 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:41.028911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1075/2133] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:31:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:31:41.071580 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d121bb-cdbd-44bc-9efc-9764e900115f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:41.072932 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-04d121bb-cdbd-44bc-9efc-9764e900115f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:41.085075 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d121bb-cdbd-44bc-9efc-9764e900115f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:41.088774 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d84b7c-dd19-4150-b492-48f3864e43ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:41.089383 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d84b7c-dd19-4150-b492-48f3864e43ed service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:41.089660 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d84b7c-dd19-4150-b492-48f3864e43ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:41.089931 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d84b7c-dd19-4150-b492-48f3864e43ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:41.146690 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d84b7c-dd19-4150-b492-48f3864e43ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:41.148201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d84b7c-dd19-4150-b492-48f3864e43ed service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:41.157083 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1059/2134] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:31:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:31:44.045108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e71d1-9379-4e4e-9497-ff48c8834919 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:44.072585 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:44.109329 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e71d1-9379-4e4e-9497-ff48c8834919 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:44.114469 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e2859-c9e5-43e4-a2b8-48cb9c287a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:44.115119 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e2859-c9e5-43e4-a2b8-48cb9c287a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:44.115393 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e2859-c9e5-43e4-a2b8-48cb9c287a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:44.115662 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e2859-c9e5-43e4-a2b8-48cb9c287a47 admin admin] PATH_INFO: `/v3/users/f644b2b3468e473faa155b402c54d7eb` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:44.116783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e2859-c9e5-43e4-a2b8-48cb9c287a47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f644b2b3468e473faa155b402c54d7eb)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:44.118658 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e2859-c9e5-43e4-a2b8-48cb9c287a47 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:44.167360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8c1e2859-c9e5-43e4-a2b8-48cb9c287a47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f644b2b3468e473faa155b402c54d7eb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.174894 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8c1e2859-c9e5-43e4-a2b8-48cb9c287a47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f644b2b3468e473faa155b402c54d7eb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.187700 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8c1e2859-c9e5-43e4-a2b8-48cb9c287a47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f644b2b3468e473faa155b402c54d7eb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.196058 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8c1e2859-c9e5-43e4-a2b8-48cb9c287a47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f644b2b3468e473faa155b402c54d7eb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.196751 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8c1e2859-c9e5-43e4-a2b8-48cb9c287a47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f644b2b3468e473faa155b402c54d7eb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.211600 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1076/2135] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:31:44 2022] DELETE /identity/v3/users/f644b2b3468e473faa155b402c54d7eb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:44.216236 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d9052-0b8b-4692-98b0-338a557184ca None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:44.216764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d9052-0b8b-4692-98b0-338a557184ca None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:44.216876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d9052-0b8b-4692-98b0-338a557184ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:44.237454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-941d9052-0b8b-4692-98b0-338a557184ca None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:44.239286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-941d9052-0b8b-4692-98b0-338a557184ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:44.301524 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1060/2136] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:44.311290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e2859-c9e5-43e4-a2b8-48cb9c287a47 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:44.312693 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8c1e2859-c9e5-43e4-a2b8-48cb9c287a47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:44.375458 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e2859-c9e5-43e4-a2b8-48cb9c287a47 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:44.379826 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48e0ce-ff98-4993-96ad-c4a32b6fc73f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:44.380452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e0ce-ff98-4993-96ad-c4a32b6fc73f service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:44.380751 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e0ce-ff98-4993-96ad-c4a32b6fc73f service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:44.381056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e0ce-ff98-4993-96ad-c4a32b6fc73f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:44.390183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48e0ce-ff98-4993-96ad-c4a32b6fc73f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:44.391670 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48e0ce-ff98-4993-96ad-c4a32b6fc73f service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:44.401639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1077/2137] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:31:44.706972 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d9052-0b8b-4692-98b0-338a557184ca None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:44.708986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-941d9052-0b8b-4692-98b0-338a557184ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:44.726526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d9052-0b8b-4692-98b0-338a557184ca None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:44.734512 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:44.735439 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:44.738833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:44.738833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] PATH_INFO: `/v3/projects/c3879c607feb4ff3a757dda8bb270dc5` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:44.738833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3879c607feb4ff3a757dda8bb270dc5)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:44.739863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:44.794537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3879c607feb4ff3a757dda8bb270dc5', 'request_id': 'req-813e7e86-14dc-413b-a5ac-924a539e9246'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.800315 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3879c607feb4ff3a757dda8bb270dc5', 'request_id': 'req-813e7e86-14dc-413b-a5ac-924a539e9246'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.807278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3879c607feb4ff3a757dda8bb270dc5', 'request_id': 'req-813e7e86-14dc-413b-a5ac-924a539e9246'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.810857 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:44.821465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] The token cache is being invalidate because project c3879c607feb4ff3a757dda8bb270dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:31:44.821721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.822443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.823911 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1061/2138] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:31:44 2022] DELETE /identity/v3/projects/c3879c607feb4ff3a757dda8bb270dc5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:44.827942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48e0ce-ff98-4993-96ad-c4a32b6fc73f service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:44.838355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48e0ce-ff98-4993-96ad-c4a32b6fc73f service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:44.882817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:44.897711 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca363a29-b819-4615-8b59-d5f4325687ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:44.898333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca363a29-b819-4615-8b59-d5f4325687ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:44.898609 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca363a29-b819-4615-8b59-d5f4325687ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:44.898879 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca363a29-b819-4615-8b59-d5f4325687ba admin admin] PATH_INFO: `/v3/users/2c28e0a7d7174e8ea0679f00fbf91dbf` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:44.900012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca363a29-b819-4615-8b59-d5f4325687ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c28e0a7d7174e8ea0679f00fbf91dbf)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:44.901915 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca363a29-b819-4615-8b59-d5f4325687ba admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:44.946033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ca363a29-b819-4615-8b59-d5f4325687ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c28e0a7d7174e8ea0679f00fbf91dbf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.953240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ca363a29-b819-4615-8b59-d5f4325687ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c28e0a7d7174e8ea0679f00fbf91dbf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.966079 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ca363a29-b819-4615-8b59-d5f4325687ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c28e0a7d7174e8ea0679f00fbf91dbf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.973326 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ca363a29-b819-4615-8b59-d5f4325687ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c28e0a7d7174e8ea0679f00fbf91dbf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.973972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ca363a29-b819-4615-8b59-d5f4325687ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c28e0a7d7174e8ea0679f00fbf91dbf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:44.979743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1078/2139] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:31:44 2022] DELETE /identity/v3/users/2c28e0a7d7174e8ea0679f00fbf91dbf => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:45.260687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:45.262400 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:45.276228 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e7e86-14dc-413b-a5ac-924a539e9246 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:45.338079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:45.338644 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:45.338954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:45.339263 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] PATH_INFO: `/v3/projects/eb236344569d45ccbd510a05586de3df` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:45.340410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb236344569d45ccbd510a05586de3df)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:45.342325 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:45.399377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb236344569d45ccbd510a05586de3df', 'request_id': 'req-73bce704-a698-4f4d-a0e7-6037c0a389d5'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:45.405977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb236344569d45ccbd510a05586de3df', 'request_id': 'req-73bce704-a698-4f4d-a0e7-6037c0a389d5'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:45.415223 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb236344569d45ccbd510a05586de3df', 'request_id': 'req-73bce704-a698-4f4d-a0e7-6037c0a389d5'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:45.430722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] The token cache is being invalidate because project eb236344569d45ccbd510a05586de3df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:31:45.431092 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:45.431883 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:45.436547 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1062/2140] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:31:45 2022] DELETE /identity/v3/projects/eb236344569d45ccbd510a05586de3df => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:45.438235 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca363a29-b819-4615-8b59-d5f4325687ba admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:45.439696 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ca363a29-b819-4615-8b59-d5f4325687ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:45.452928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca363a29-b819-4615-8b59-d5f4325687ba admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:45.510299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554b25e-0887-4d39-a69e-01ef96491b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:45.510930 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554b25e-0887-4d39-a69e-01ef96491b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:45.511239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554b25e-0887-4d39-a69e-01ef96491b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:45.511511 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554b25e-0887-4d39-a69e-01ef96491b93 admin admin] PATH_INFO: `/v3/users/1fe30007bf8747d68ad619c000661fb0` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:45.512764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554b25e-0887-4d39-a69e-01ef96491b93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fe30007bf8747d68ad619c000661fb0)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:45.515996 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554b25e-0887-4d39-a69e-01ef96491b93 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:45.577184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a554b25e-0887-4d39-a69e-01ef96491b93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fe30007bf8747d68ad619c000661fb0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:45.586430 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a554b25e-0887-4d39-a69e-01ef96491b93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fe30007bf8747d68ad619c000661fb0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:45.599159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:45.601628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a554b25e-0887-4d39-a69e-01ef96491b93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fe30007bf8747d68ad619c000661fb0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:45.612216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a554b25e-0887-4d39-a69e-01ef96491b93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fe30007bf8747d68ad619c000661fb0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:45.612741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a554b25e-0887-4d39-a69e-01ef96491b93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fe30007bf8747d68ad619c000661fb0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:45.620269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1079/2141] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:31:45 2022] DELETE /identity/v3/users/1fe30007bf8747d68ad619c000661fb0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:45.891166 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:45.892968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:45.906862 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bce704-a698-4f4d-a0e7-6037c0a389d5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:45.964481 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a3763-2840-4ff1-8d7e-340a4b8884fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:45.965121 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a3763-2840-4ff1-8d7e-340a4b8884fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:45.965398 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a3763-2840-4ff1-8d7e-340a4b8884fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:45.965668 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a3763-2840-4ff1-8d7e-340a4b8884fa admin admin] PATH_INFO: `/v3/projects/b33fa8ce9aa54b889d66e57de508821d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:45.966780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a3763-2840-4ff1-8d7e-340a4b8884fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b33fa8ce9aa54b889d66e57de508821d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:45.968726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a3763-2840-4ff1-8d7e-340a4b8884fa admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:45.998423 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:46.016081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-762a3763-2840-4ff1-8d7e-340a4b8884fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b33fa8ce9aa54b889d66e57de508821d', 'request_id': 'req-762a3763-2840-4ff1-8d7e-340a4b8884fa'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:46.022250 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-762a3763-2840-4ff1-8d7e-340a4b8884fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b33fa8ce9aa54b889d66e57de508821d', 'request_id': 'req-762a3763-2840-4ff1-8d7e-340a4b8884fa'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:46.028269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-762a3763-2840-4ff1-8d7e-340a4b8884fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b33fa8ce9aa54b889d66e57de508821d', 'request_id': 'req-762a3763-2840-4ff1-8d7e-340a4b8884fa'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:46.040907 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-762a3763-2840-4ff1-8d7e-340a4b8884fa admin admin] The token cache is being invalidate because project b33fa8ce9aa54b889d66e57de508821d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:31:46.041195 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-762a3763-2840-4ff1-8d7e-340a4b8884fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:46.041842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-762a3763-2840-4ff1-8d7e-340a4b8884fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:46.043357 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1063/2142] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:31:45 2022] DELETE /identity/v3/projects/b33fa8ce9aa54b889d66e57de508821d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:46.049184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb83eb-07e2-4aa1-bc40-37fdaf635e4a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:46.049468 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb83eb-07e2-4aa1-bc40-37fdaf635e4a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:46.049741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb83eb-07e2-4aa1-bc40-37fdaf635e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:46.070154 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bcfb83eb-07e2-4aa1-bc40-37fdaf635e4a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:46.071383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bcfb83eb-07e2-4aa1-bc40-37fdaf635e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:46.130385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1080/2143] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:46.139942 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21eed2-a70d-4db6-a6bf-fce8c565d67a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:46.140226 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21eed2-a70d-4db6-a6bf-fce8c565d67a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:46.140503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21eed2-a70d-4db6-a6bf-fce8c565d67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:46.160324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9e21eed2-a70d-4db6-a6bf-fce8c565d67a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:46.161518 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9e21eed2-a70d-4db6-a6bf-fce8c565d67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:46.169356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1064/2144] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:46.173657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb83eb-07e2-4aa1-bc40-37fdaf635e4a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:46.183260 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb83eb-07e2-4aa1-bc40-37fdaf635e4a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:46.187744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed6fd7-a6bc-4142-9017-fe89934d639f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:46.188333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed6fd7-a6bc-4142-9017-fe89934d639f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:46.188625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed6fd7-a6bc-4142-9017-fe89934d639f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:46.188893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed6fd7-a6bc-4142-9017-fe89934d639f admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:46.189527 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed6fd7-a6bc-4142-9017-fe89934d639f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:31:46.189825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed6fd7-a6bc-4142-9017-fe89934d639f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:46.192003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed6fd7-a6bc-4142-9017-fe89934d639f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:46.203168 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1081/2145] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:31:46.208133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21eed2-a70d-4db6-a6bf-fce8c565d67a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:46.217228 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21eed2-a70d-4db6-a6bf-fce8c565d67a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:46.221837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e8682-89d8-4257-93b9-19b622fb320e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:46.222432 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e8682-89d8-4257-93b9-19b622fb320e admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:46.222705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e8682-89d8-4257-93b9-19b622fb320e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:46.222970 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e8682-89d8-4257-93b9-19b622fb320e admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:46.223631 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e8682-89d8-4257-93b9-19b622fb320e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:46.225502 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e8682-89d8-4257-93b9-19b622fb320e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:46.251617 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1065/2146] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:31:46 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:31:46.256266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed6fd7-a6bc-4142-9017-fe89934d639f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:46.266346 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed6fd7-a6bc-4142-9017-fe89934d639f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:46.321630 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55136651-7f73-4146-b3e3-994aa2a3fa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:46.322225 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55136651-7f73-4146-b3e3-994aa2a3fa48 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:46.322499 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55136651-7f73-4146-b3e3-994aa2a3fa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:46.322764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55136651-7f73-4146-b3e3-994aa2a3fa48 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:46.323415 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55136651-7f73-4146-b3e3-994aa2a3fa48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:46.325303 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55136651-7f73-4146-b3e3-994aa2a3fa48 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:46.346375 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1082/2147] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:31:46 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:31:46.351148 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e8682-89d8-4257-93b9-19b622fb320e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:46.362567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e8682-89d8-4257-93b9-19b622fb320e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:46.368062 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65c914-3028-449c-84ab-bac73a79ac64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:46.368667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65c914-3028-449c-84ab-bac73a79ac64 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:46.368947 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65c914-3028-449c-84ab-bac73a79ac64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:46.369221 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65c914-3028-449c-84ab-bac73a79ac64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:46.369912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65c914-3028-449c-84ab-bac73a79ac64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:46.371564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65c914-3028-449c-84ab-bac73a79ac64 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:46.382162 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1066/2148] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:31:46.386121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55136651-7f73-4146-b3e3-994aa2a3fa48 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:46.395997 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55136651-7f73-4146-b3e3-994aa2a3fa48 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:46.400536 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa238458-6229-49a0-862d-085f39423b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:46.401129 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238458-6229-49a0-862d-085f39423b64 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:46.401430 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238458-6229-49a0-862d-085f39423b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:46.401704 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238458-6229-49a0-862d-085f39423b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:46.402343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa238458-6229-49a0-862d-085f39423b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:46.403970 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa238458-6229-49a0-862d-085f39423b64 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:46.414191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1083/2149] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:31:46.418167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65c914-3028-449c-84ab-bac73a79ac64 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:46.428554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65c914-3028-449c-84ab-bac73a79ac64 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:46.433080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241abfb7-79bd-4712-b870-886c6f395087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:46.433721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241abfb7-79bd-4712-b870-886c6f395087 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:46.433994 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241abfb7-79bd-4712-b870-886c6f395087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:46.434261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241abfb7-79bd-4712-b870-886c6f395087 admin admin] PATH_INFO: `/v3/projects/dc20e3921bcb4fd88809021eb57aed9f/users/04eb2af43a544a50810f368746ee5c15/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:46.449591 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241abfb7-79bd-4712-b870-886c6f395087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc20e3921bcb4fd88809021eb57aed9f, user_id=04eb2af43a544a50810f368746ee5c15, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:46.452274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241abfb7-79bd-4712-b870-886c6f395087 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:46.465698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1067/2150] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:31:46 2022] PUT /identity/v3/projects/dc20e3921bcb4fd88809021eb57aed9f/users/04eb2af43a544a50810f368746ee5c15/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:46.477506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b963895-9cbe-4a22-b4f0-f0e95a755016 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:46.477979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b963895-9cbe-4a22-b4f0-f0e95a755016 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:46.478104 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b963895-9cbe-4a22-b4f0-f0e95a755016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:46.520857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1b963895-9cbe-4a22-b4f0-f0e95a755016 None None] MFA Rules not processed for user `04eb2af43a544a50810f368746ee5c15`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:46.522534 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1b963895-9cbe-4a22-b4f0-f0e95a755016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:46.575529 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:46.602957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1084/2151] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:46.616391 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241abfb7-79bd-4712-b870-886c6f395087 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:46.621357 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-241abfb7-79bd-4712-b870-886c6f395087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:46.695010 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241abfb7-79bd-4712-b870-886c6f395087 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:46.698921 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717582c2-96e2-45f1-8f8b-30d5de466389 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:46.700231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717582c2-96e2-45f1-8f8b-30d5de466389 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:46.700231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717582c2-96e2-45f1-8f8b-30d5de466389 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:46.700490 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717582c2-96e2-45f1-8f8b-30d5de466389 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:46.709200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717582c2-96e2-45f1-8f8b-30d5de466389 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:46.710747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717582c2-96e2-45f1-8f8b-30d5de466389 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:46.719901 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1068/2152] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:31:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:31:48.052299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b963895-9cbe-4a22-b4f0-f0e95a755016 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:48.053805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1b963895-9cbe-4a22-b4f0-f0e95a755016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:48.124059 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b963895-9cbe-4a22-b4f0-f0e95a755016 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:48.127986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617b87c-c8c4-429a-b96a-07853473d410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:48.129219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617b87c-c8c4-429a-b96a-07853473d410 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:48.129756 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617b87c-c8c4-429a-b96a-07853473d410 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:48.129885 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617b87c-c8c4-429a-b96a-07853473d410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:48.133709 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f617b87c-c8c4-429a-b96a-07853473d410 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:48.233261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617b87c-c8c4-429a-b96a-07853473d410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:48.234981 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617b87c-c8c4-429a-b96a-07853473d410 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:48.246689 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1085/2153] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:31:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:31:49.820291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717582c2-96e2-45f1-8f8b-30d5de466389 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:49.887645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:49.898463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717582c2-96e2-45f1-8f8b-30d5de466389 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:49.905842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d2472-a71f-44f0-9a2d-e5bc7c8d2e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:49.906500 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d2472-a71f-44f0-9a2d-e5bc7c8d2e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:49.906783 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d2472-a71f-44f0-9a2d-e5bc7c8d2e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:49.907229 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d2472-a71f-44f0-9a2d-e5bc7c8d2e07 admin admin] PATH_INFO: `/v3/users/04eb2af43a544a50810f368746ee5c15` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:49.908392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d2472-a71f-44f0-9a2d-e5bc7c8d2e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04eb2af43a544a50810f368746ee5c15)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:49.911351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d2472-a71f-44f0-9a2d-e5bc7c8d2e07 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:49.974682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6f0d2472-a71f-44f0-9a2d-e5bc7c8d2e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04eb2af43a544a50810f368746ee5c15'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:49.984676 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6f0d2472-a71f-44f0-9a2d-e5bc7c8d2e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04eb2af43a544a50810f368746ee5c15'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:49.985551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6f0d2472-a71f-44f0-9a2d-e5bc7c8d2e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04eb2af43a544a50810f368746ee5c15'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:50.000129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6f0d2472-a71f-44f0-9a2d-e5bc7c8d2e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04eb2af43a544a50810f368746ee5c15'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:50.007808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6f0d2472-a71f-44f0-9a2d-e5bc7c8d2e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04eb2af43a544a50810f368746ee5c15'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:50.017417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1069/2154] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:31:49 2022] DELETE /identity/v3/users/04eb2af43a544a50810f368746ee5c15 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:50.019110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bbccb-66f9-43bc-a018-664539f51fd4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:50.019749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bbccb-66f9-43bc-a018-664539f51fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:50.019749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bbccb-66f9-43bc-a018-664539f51fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:50.043180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5c5bbccb-66f9-43bc-a018-664539f51fd4 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:50.045440 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5c5bbccb-66f9-43bc-a018-664539f51fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:50.117066 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1086/2155] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:50.128409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d2472-a71f-44f0-9a2d-e5bc7c8d2e07 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:50.130114 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6f0d2472-a71f-44f0-9a2d-e5bc7c8d2e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:50.201735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d2472-a71f-44f0-9a2d-e5bc7c8d2e07 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:50.207176 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631565fc-1660-41be-99ad-2b00c537b5a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:50.207896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631565fc-1660-41be-99ad-2b00c537b5a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:50.208279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631565fc-1660-41be-99ad-2b00c537b5a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:50.208625 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631565fc-1660-41be-99ad-2b00c537b5a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:50.221471 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631565fc-1660-41be-99ad-2b00c537b5a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:50.223706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631565fc-1660-41be-99ad-2b00c537b5a5 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:50.235040 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1070/2156] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:31:50.890145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bbccb-66f9-43bc-a018-664539f51fd4 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:50.891943 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5c5bbccb-66f9-43bc-a018-664539f51fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:50.927833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bbccb-66f9-43bc-a018-664539f51fd4 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:50.942817 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c7fea-813a-43d6-853d-a04c3181c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:50.943603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c7fea-813a-43d6-853d-a04c3181c3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:50.943935 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c7fea-813a-43d6-853d-a04c3181c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:50.944210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c7fea-813a-43d6-853d-a04c3181c3ad admin admin] PATH_INFO: `/v3/projects/dc20e3921bcb4fd88809021eb57aed9f` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:50.945905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c7fea-813a-43d6-853d-a04c3181c3ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc20e3921bcb4fd88809021eb57aed9f)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:50.948102 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c7fea-813a-43d6-853d-a04c3181c3ad admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:51.025985 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:51.061273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-592c7fea-813a-43d6-853d-a04c3181c3ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc20e3921bcb4fd88809021eb57aed9f', 'request_id': 'req-592c7fea-813a-43d6-853d-a04c3181c3ad'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:51.071440 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-592c7fea-813a-43d6-853d-a04c3181c3ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc20e3921bcb4fd88809021eb57aed9f', 'request_id': 'req-592c7fea-813a-43d6-853d-a04c3181c3ad'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:51.112155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-592c7fea-813a-43d6-853d-a04c3181c3ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc20e3921bcb4fd88809021eb57aed9f', 'request_id': 'req-592c7fea-813a-43d6-853d-a04c3181c3ad'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:51.137309 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-592c7fea-813a-43d6-853d-a04c3181c3ad admin admin] The token cache is being invalidate because project dc20e3921bcb4fd88809021eb57aed9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:31:51.137625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-592c7fea-813a-43d6-853d-a04c3181c3ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:51.138260 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-592c7fea-813a-43d6-853d-a04c3181c3ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:31:51.142940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1087/2157] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:31:50 2022] DELETE /identity/v3/projects/dc20e3921bcb4fd88809021eb57aed9f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:31:51.145021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26124ea-62ff-4d4a-ab7d-ee41a17b5299 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:51.145478 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26124ea-62ff-4d4a-ab7d-ee41a17b5299 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:51.145755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26124ea-62ff-4d4a-ab7d-ee41a17b5299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:51.174032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a26124ea-62ff-4d4a-ab7d-ee41a17b5299 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:51.176387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a26124ea-62ff-4d4a-ab7d-ee41a17b5299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:51.259662 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1071/2158] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:31:51.261387 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1fea5-a422-41c3-8946-18bf8f8339ca None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:51.261724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1fea5-a422-41c3-8946-18bf8f8339ca None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:51.262166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1fea5-a422-41c3-8946-18bf8f8339ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:51.286394 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-47f1fea5-a422-41c3-8946-18bf8f8339ca None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:51.287668 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-47f1fea5-a422-41c3-8946-18bf8f8339ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:51.298857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1088/2159] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:51.303737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26124ea-62ff-4d4a-ab7d-ee41a17b5299 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:51.317645 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26124ea-62ff-4d4a-ab7d-ee41a17b5299 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:51.323639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6d6f2-0b5c-479b-a15c-5c7a4730a982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:51.324266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6d6f2-0b5c-479b-a15c-5c7a4730a982 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:51.324546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6d6f2-0b5c-479b-a15c-5c7a4730a982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:51.324813 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6d6f2-0b5c-479b-a15c-5c7a4730a982 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:51.325467 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6d6f2-0b5c-479b-a15c-5c7a4730a982 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:31:51.325765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6d6f2-0b5c-479b-a15c-5c7a4730a982 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:51.327922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6d6f2-0b5c-479b-a15c-5c7a4730a982 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:51.340999 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1072/2160] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:31:51.348451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1fea5-a422-41c3-8946-18bf8f8339ca None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:51.358553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1fea5-a422-41c3-8946-18bf8f8339ca None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:51.365244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9809f8e-c207-47d9-9782-9afb74232261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:51.365927 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9809f8e-c207-47d9-9782-9afb74232261 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:51.366279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9809f8e-c207-47d9-9782-9afb74232261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:51.366625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9809f8e-c207-47d9-9782-9afb74232261 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:51.368063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9809f8e-c207-47d9-9782-9afb74232261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:51.369731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9809f8e-c207-47d9-9782-9afb74232261 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:51.402385 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1089/2161] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:31:51 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:31:51.406079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6d6f2-0b5c-479b-a15c-5c7a4730a982 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:51.414734 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:51.419152 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6d6f2-0b5c-479b-a15c-5c7a4730a982 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:51.500033 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d88c4-2464-42ab-972c-2dd6c5664844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:51.500909 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d88c4-2464-42ab-972c-2dd6c5664844 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:51.501191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d88c4-2464-42ab-972c-2dd6c5664844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:51.501463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d88c4-2464-42ab-972c-2dd6c5664844 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:51.502106 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d88c4-2464-42ab-972c-2dd6c5664844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:51.504601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d88c4-2464-42ab-972c-2dd6c5664844 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:51.548154 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1073/2162] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:31:51 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:31:51.548647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9809f8e-c207-47d9-9782-9afb74232261 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:51.559507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9809f8e-c207-47d9-9782-9afb74232261 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:51.565379 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c95021-baa1-4338-8b95-e31ef90f4ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:51.565379 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c95021-baa1-4338-8b95-e31ef90f4ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:51.566193 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c95021-baa1-4338-8b95-e31ef90f4ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:51.566293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c95021-baa1-4338-8b95-e31ef90f4ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:51.566929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c95021-baa1-4338-8b95-e31ef90f4ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:51.568639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c95021-baa1-4338-8b95-e31ef90f4ae8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:51.583384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1090/2163] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:31:51.584918 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d88c4-2464-42ab-972c-2dd6c5664844 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:51.595812 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d88c4-2464-42ab-972c-2dd6c5664844 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:51.603492 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393d627-0c9a-497e-82a2-ec88b2b761cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:51.604640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393d627-0c9a-497e-82a2-ec88b2b761cf admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:51.604640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393d627-0c9a-497e-82a2-ec88b2b761cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:51.605051 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393d627-0c9a-497e-82a2-ec88b2b761cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:51.605839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393d627-0c9a-497e-82a2-ec88b2b761cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:51.607954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393d627-0c9a-497e-82a2-ec88b2b761cf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:51.622641 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1074/2164] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:31:51.626593 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c95021-baa1-4338-8b95-e31ef90f4ae8 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:51.638446 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c95021-baa1-4338-8b95-e31ef90f4ae8 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:51.643519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b54f2-4310-42f1-92df-e645ff7bfc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:51.643905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b54f2-4310-42f1-92df-e645ff7bfc7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:51.644183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b54f2-4310-42f1-92df-e645ff7bfc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:51.644525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b54f2-4310-42f1-92df-e645ff7bfc7f admin admin] PATH_INFO: `/v3/projects/1147d14fb7b5456382c0097ecd3f66b4/users/feedcd9bb713407c89d0ee9955c6eb38/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:51.664530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b54f2-4310-42f1-92df-e645ff7bfc7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1147d14fb7b5456382c0097ecd3f66b4, user_id=feedcd9bb713407c89d0ee9955c6eb38, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:51.667481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b54f2-4310-42f1-92df-e645ff7bfc7f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:51.684595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1091/2165] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:31:51 2022] PUT /identity/v3/projects/1147d14fb7b5456382c0097ecd3f66b4/users/feedcd9bb713407c89d0ee9955c6eb38/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:31:51.762074 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1981e-863b-4aad-90c2-62082add4d64 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:51.762293 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1981e-863b-4aad-90c2-62082add4d64 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:51.762555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1981e-863b-4aad-90c2-62082add4d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:51.805460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-84e1981e-863b-4aad-90c2-62082add4d64 None None] MFA Rules not processed for user `feedcd9bb713407c89d0ee9955c6eb38`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:31:51.807344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-84e1981e-863b-4aad-90c2-62082add4d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:51.902942 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1075/2166] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:31:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:31:51.914729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b54f2-4310-42f1-92df-e645ff7bfc7f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:51.916734 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e15b54f2-4310-42f1-92df-e645ff7bfc7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:51.989828 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b54f2-4310-42f1-92df-e645ff7bfc7f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:51.993428 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f5c58-fd80-4903-9bba-8fa8805affac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:51.994043 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f5c58-fd80-4903-9bba-8fa8805affac service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:51.994316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f5c58-fd80-4903-9bba-8fa8805affac service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:51.994583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f5c58-fd80-4903-9bba-8fa8805affac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:52.004661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f5c58-fd80-4903-9bba-8fa8805affac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:52.006182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f5c58-fd80-4903-9bba-8fa8805affac service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:52.016262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1092/2167] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:31:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:31:57.264436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1981e-863b-4aad-90c2-62082add4d64 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:57.265805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-84e1981e-863b-4aad-90c2-62082add4d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:57.324205 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1981e-863b-4aad-90c2-62082add4d64 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:57.327723 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cc19d-b786-4032-83e3-e182735b2038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:57.328361 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cc19d-b786-4032-83e3-e182735b2038 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:57.328633 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cc19d-b786-4032-83e3-e182735b2038 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:57.328902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cc19d-b786-4032-83e3-e182735b2038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:57.330617 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-542cc19d-b786-4032-83e3-e182735b2038 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:57.360586 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:57.398283 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cc19d-b786-4032-83e3-e182735b2038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:31:57.399764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cc19d-b786-4032-83e3-e182735b2038 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:31:57.410198 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1076/2168] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:31:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:31:59.833347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f5c58-fd80-4903-9bba-8fa8805affac service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:31:59.835139 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-310f5c58-fd80-4903-9bba-8fa8805affac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:31:59.897184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:31:59.920893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f5c58-fd80-4903-9bba-8fa8805affac service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:31:59.927059 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56527c9e-0db8-4cb4-994e-8add46a54972 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:31:59.928242 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56527c9e-0db8-4cb4-994e-8add46a54972 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:31:59.928242 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56527c9e-0db8-4cb4-994e-8add46a54972 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:31:59.928619 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56527c9e-0db8-4cb4-994e-8add46a54972 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:31:59.931051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-56527c9e-0db8-4cb4-994e-8add46a54972 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:00.006645 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56527c9e-0db8-4cb4-994e-8add46a54972 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:00.008604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56527c9e-0db8-4cb4-994e-8add46a54972 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:00.021836 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1093/2169] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:31:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:00.730859 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cc19d-b786-4032-83e3-e182735b2038 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:00.732605 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-542cc19d-b786-4032-83e3-e182735b2038 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:00.793815 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56527c9e-0db8-4cb4-994e-8add46a54972 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:00.795418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-56527c9e-0db8-4cb4-994e-8add46a54972 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:00.813211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56527c9e-0db8-4cb4-994e-8add46a54972 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:00.814026 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cc19d-b786-4032-83e3-e182735b2038 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:00.818082 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c759b9-8a5b-4ecf-aceb-33136888c3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:00.818716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c759b9-8a5b-4ecf-aceb-33136888c3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:00.819001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c759b9-8a5b-4ecf-aceb-33136888c3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:00.819322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c759b9-8a5b-4ecf-aceb-33136888c3ac admin admin] PATH_INFO: `/v3/users/7d4748cbe63d490b9d8b6914a27ff364` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:00.819452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90aac92-75a8-4d7d-9b40-d0bc1a78cffe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e935ba63cdd4256874b99a2eb0a1a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:00.820068 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90aac92-75a8-4d7d-9b40-d0bc1a78cffe service placement] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:00.820375 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90aac92-75a8-4d7d-9b40-d0bc1a78cffe service placement] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:00.820477 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c759b9-8a5b-4ecf-aceb-33136888c3ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d4748cbe63d490b9d8b6914a27ff364)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:00.820649 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90aac92-75a8-4d7d-9b40-d0bc1a78cffe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:00.822367 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d90aac92-75a8-4d7d-9b40-d0bc1a78cffe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:00.822475 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c759b9-8a5b-4ecf-aceb-33136888c3ac admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:00.838034 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90aac92-75a8-4d7d-9b40-d0bc1a78cffe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:00.839675 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90aac92-75a8-4d7d-9b40-d0bc1a78cffe service placement] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:00.852675 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1077/2170] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:00.881594 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-48c759b9-8a5b-4ecf-aceb-33136888c3ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d4748cbe63d490b9d8b6914a27ff364'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:00.889566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-48c759b9-8a5b-4ecf-aceb-33136888c3ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d4748cbe63d490b9d8b6914a27ff364'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:00.902329 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-48c759b9-8a5b-4ecf-aceb-33136888c3ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d4748cbe63d490b9d8b6914a27ff364'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:00.910066 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-48c759b9-8a5b-4ecf-aceb-33136888c3ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d4748cbe63d490b9d8b6914a27ff364'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:00.910663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-48c759b9-8a5b-4ecf-aceb-33136888c3ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d4748cbe63d490b9d8b6914a27ff364'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:00.916560 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1094/2171] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:00 2022] DELETE /identity/v3/users/7d4748cbe63d490b9d8b6914a27ff364 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:01.086805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90aac92-75a8-4d7d-9b40-d0bc1a78cffe service placement] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:01.099483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90aac92-75a8-4d7d-9b40-d0bc1a78cffe service placement] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:01.166831 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:01.167495 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:01.167769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:01.168039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff admin admin] PATH_INFO: `/v3/projects/1d2f70db22ec43d5ba9125f940433320` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:01.169149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d2f70db22ec43d5ba9125f940433320)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:01.171039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:01.218468 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d2f70db22ec43d5ba9125f940433320', 'request_id': 'req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:01.226537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:01.228899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d2f70db22ec43d5ba9125f940433320', 'request_id': 'req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:01.236342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d2f70db22ec43d5ba9125f940433320', 'request_id': 'req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:01.249147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff admin admin] The token cache is being invalidate because project 1d2f70db22ec43d5ba9125f940433320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:01.249520 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:01.250045 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8cd7c178-9390-43cf-8e6f-dd27f161a9ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:01.251557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1078/2172] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:01 2022] DELETE /identity/v3/projects/1d2f70db22ec43d5ba9125f940433320 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:01.256902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b4c71-1ed6-4a2e-8652-4e889d1eb518 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:01.257182 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b4c71-1ed6-4a2e-8652-4e889d1eb518 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:01.257453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b4c71-1ed6-4a2e-8652-4e889d1eb518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:01.277351 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-325b4c71-1ed6-4a2e-8652-4e889d1eb518 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:01.278540 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-325b4c71-1ed6-4a2e-8652-4e889d1eb518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:01.346136 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1095/2173] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:01.354045 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95a71e-ea4b-4097-879f-1bf834b1d15f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:01.354329 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95a71e-ea4b-4097-879f-1bf834b1d15f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:01.354605 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95a71e-ea4b-4097-879f-1bf834b1d15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:01.375115 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-0c95a71e-ea4b-4097-879f-1bf834b1d15f None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:01.376292 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0c95a71e-ea4b-4097-879f-1bf834b1d15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:01.385361 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1079/2174] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:01.389484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b4c71-1ed6-4a2e-8652-4e889d1eb518 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:01.394402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:01.401002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b4c71-1ed6-4a2e-8652-4e889d1eb518 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:01.406481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d20b45-20ad-482f-9160-f08e7738f3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:01.407108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d20b45-20ad-482f-9160-f08e7738f3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:01.407395 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d20b45-20ad-482f-9160-f08e7738f3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:01.407669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d20b45-20ad-482f-9160-f08e7738f3d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:01.408318 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d20b45-20ad-482f-9160-f08e7738f3d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:32:01.408625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d20b45-20ad-482f-9160-f08e7738f3d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:01.410226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d20b45-20ad-482f-9160-f08e7738f3d4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:01.421588 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1096/2175] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:32:01.426563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95a71e-ea4b-4097-879f-1bf834b1d15f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:01.435900 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95a71e-ea4b-4097-879f-1bf834b1d15f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:01.441320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbef20d-a590-48da-9290-6cde457d908a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:01.441960 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbef20d-a590-48da-9290-6cde457d908a admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:01.442235 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbef20d-a590-48da-9290-6cde457d908a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:01.442500 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbef20d-a590-48da-9290-6cde457d908a admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:01.443200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbef20d-a590-48da-9290-6cde457d908a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:01.445151 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbef20d-a590-48da-9290-6cde457d908a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:01.472475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1080/2176] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:01 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:01.476738 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d20b45-20ad-482f-9160-f08e7738f3d4 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:01.488128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d20b45-20ad-482f-9160-f08e7738f3d4 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:01.568507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9b27e-a940-4531-8228-dafe1ed93998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:01.568507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9b27e-a940-4531-8228-dafe1ed93998 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:01.568507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9b27e-a940-4531-8228-dafe1ed93998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:01.568507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9b27e-a940-4531-8228-dafe1ed93998 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:01.571736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9b27e-a940-4531-8228-dafe1ed93998 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:01.571736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9b27e-a940-4531-8228-dafe1ed93998 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:01.590549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1097/2177] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:01 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:01.591836 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbef20d-a590-48da-9290-6cde457d908a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:01.602777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbef20d-a590-48da-9290-6cde457d908a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:01.607747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca94f8b-4772-44db-b85e-dc9d8c3abb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:01.608348 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca94f8b-4772-44db-b85e-dc9d8c3abb4e admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:01.608627 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca94f8b-4772-44db-b85e-dc9d8c3abb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:01.608900 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca94f8b-4772-44db-b85e-dc9d8c3abb4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:01.609540 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca94f8b-4772-44db-b85e-dc9d8c3abb4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:01.611167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca94f8b-4772-44db-b85e-dc9d8c3abb4e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:01.621574 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1081/2178] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:01.625774 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9b27e-a940-4531-8228-dafe1ed93998 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:01.636199 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9b27e-a940-4531-8228-dafe1ed93998 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:01.640969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f10e34-79d7-488c-acbf-6637f8b9a36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:01.641574 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f10e34-79d7-488c-acbf-6637f8b9a36a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:01.641855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f10e34-79d7-488c-acbf-6637f8b9a36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:01.642131 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f10e34-79d7-488c-acbf-6637f8b9a36a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:01.643567 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f10e34-79d7-488c-acbf-6637f8b9a36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:01.645367 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f10e34-79d7-488c-acbf-6637f8b9a36a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:01.656445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1098/2179] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:01.661721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca94f8b-4772-44db-b85e-dc9d8c3abb4e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:01.672270 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca94f8b-4772-44db-b85e-dc9d8c3abb4e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:01.677245 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e934a-c10e-425f-a376-6ec7e8eada40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:01.677911 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e934a-c10e-425f-a376-6ec7e8eada40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:01.678190 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e934a-c10e-425f-a376-6ec7e8eada40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:01.678464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e934a-c10e-425f-a376-6ec7e8eada40 admin admin] PATH_INFO: `/v3/projects/1702fd082d2c42cda1f04b2f1299b76f/users/fd3ee784609540cbba19adb3adb38bab/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:01.694270 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e934a-c10e-425f-a376-6ec7e8eada40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1702fd082d2c42cda1f04b2f1299b76f, user_id=fd3ee784609540cbba19adb3adb38bab, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:01.697131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e934a-c10e-425f-a376-6ec7e8eada40 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:01.713773 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1082/2180] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:01 2022] PUT /identity/v3/projects/1702fd082d2c42cda1f04b2f1299b76f/users/fd3ee784609540cbba19adb3adb38bab/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:01.726031 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a2725-0274-4a23-ba8e-dfbb03ff61db None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:01.726418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a2725-0274-4a23-ba8e-dfbb03ff61db None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:01.726831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a2725-0274-4a23-ba8e-dfbb03ff61db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:01.757428 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-430a2725-0274-4a23-ba8e-dfbb03ff61db None None] MFA Rules not processed for user `fd3ee784609540cbba19adb3adb38bab`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:01.758654 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-430a2725-0274-4a23-ba8e-dfbb03ff61db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:01.825424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1099/2181] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:01.830214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e934a-c10e-425f-a376-6ec7e8eada40 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:01.913084 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e934a-c10e-425f-a376-6ec7e8eada40 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:01.918451 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a94485-b550-41f4-8671-bb3d7e27b491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:01.919074 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a94485-b550-41f4-8671-bb3d7e27b491 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:01.919497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a94485-b550-41f4-8671-bb3d7e27b491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:01.919772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a94485-b550-41f4-8671-bb3d7e27b491 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:01.920409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a94485-b550-41f4-8671-bb3d7e27b491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:01.922400 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a94485-b550-41f4-8671-bb3d7e27b491 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:01.949699 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1083/2182] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:01 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:01.953855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a2725-0274-4a23-ba8e-dfbb03ff61db None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:01.965142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a2725-0274-4a23-ba8e-dfbb03ff61db None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:02.019425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:02.026924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19662969-1230-4faa-b7e9-f41e73c6ae8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:02.027556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19662969-1230-4faa-b7e9-f41e73c6ae8d admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:02.027831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19662969-1230-4faa-b7e9-f41e73c6ae8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:02.028099 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19662969-1230-4faa-b7e9-f41e73c6ae8d admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:02.028723 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19662969-1230-4faa-b7e9-f41e73c6ae8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:02.030680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19662969-1230-4faa-b7e9-f41e73c6ae8d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:02.052516 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1100/2183] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:01 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:02.056574 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a94485-b550-41f4-8671-bb3d7e27b491 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:02.067574 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a94485-b550-41f4-8671-bb3d7e27b491 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:02.072183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b3e45-9a81-4101-ae19-02ebbacabb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:02.072778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b3e45-9a81-4101-ae19-02ebbacabb70 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:02.073055 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b3e45-9a81-4101-ae19-02ebbacabb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:02.073325 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b3e45-9a81-4101-ae19-02ebbacabb70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:02.073958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b3e45-9a81-4101-ae19-02ebbacabb70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:02.075568 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b3e45-9a81-4101-ae19-02ebbacabb70 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:02.086788 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1084/2184] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:02.090868 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19662969-1230-4faa-b7e9-f41e73c6ae8d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:02.100756 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19662969-1230-4faa-b7e9-f41e73c6ae8d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:02.105277 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39202b88-3fc0-42dc-924e-f1ea74530560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:02.105921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39202b88-3fc0-42dc-924e-f1ea74530560 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:02.106199 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39202b88-3fc0-42dc-924e-f1ea74530560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:02.106475 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39202b88-3fc0-42dc-924e-f1ea74530560 admin admin] PATH_INFO: `/v3/projects/3043378b75e94a0f88fbe3dd06970466/users/0622e87436514c02a5726afe30011973/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:02.121942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39202b88-3fc0-42dc-924e-f1ea74530560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3043378b75e94a0f88fbe3dd06970466, user_id=0622e87436514c02a5726afe30011973, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:02.124647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39202b88-3fc0-42dc-924e-f1ea74530560 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:02.138051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1101/2185] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:02 2022] PUT /identity/v3/projects/3043378b75e94a0f88fbe3dd06970466/users/0622e87436514c02a5726afe30011973/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:02.141852 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b3e45-9a81-4101-ae19-02ebbacabb70 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:02.152173 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b3e45-9a81-4101-ae19-02ebbacabb70 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:02.201539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:02.212497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99834e50-70e6-4f84-bfba-05d2a97255a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:02.213114 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99834e50-70e6-4f84-bfba-05d2a97255a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:02.213392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99834e50-70e6-4f84-bfba-05d2a97255a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:02.213663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99834e50-70e6-4f84-bfba-05d2a97255a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:02.214359 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99834e50-70e6-4f84-bfba-05d2a97255a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:02.216104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99834e50-70e6-4f84-bfba-05d2a97255a4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:02.226190 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1085/2186] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:02.230815 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39202b88-3fc0-42dc-924e-f1ea74530560 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:02.241805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39202b88-3fc0-42dc-924e-f1ea74530560 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:02.246375 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98420a4c-5537-436b-9865-e7fd956ff932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:02.246963 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98420a4c-5537-436b-9865-e7fd956ff932 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:02.247266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98420a4c-5537-436b-9865-e7fd956ff932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:02.247537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98420a4c-5537-436b-9865-e7fd956ff932 admin admin] PATH_INFO: `/v3/projects/3043378b75e94a0f88fbe3dd06970466/users/0622e87436514c02a5726afe30011973/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:02.249396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98420a4c-5537-436b-9865-e7fd956ff932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3043378b75e94a0f88fbe3dd06970466, user_id=0622e87436514c02a5726afe30011973, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:02.252113 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98420a4c-5537-436b-9865-e7fd956ff932 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:02.262924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1102/2187] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:02 2022] PUT /identity/v3/projects/3043378b75e94a0f88fbe3dd06970466/users/0622e87436514c02a5726afe30011973/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:02.273104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b86471-15b7-43de-9b83-66f26435f31f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:02.273432 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b86471-15b7-43de-9b83-66f26435f31f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:02.273734 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b86471-15b7-43de-9b83-66f26435f31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:02.304805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-22b86471-15b7-43de-9b83-66f26435f31f None None] MFA Rules not processed for user `0622e87436514c02a5726afe30011973`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:02.306286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-22b86471-15b7-43de-9b83-66f26435f31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:02.376868 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1086/2188] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:02.379010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98420a4c-5537-436b-9865-e7fd956ff932 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:02.380845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-98420a4c-5537-436b-9865-e7fd956ff932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:02.387443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b86471-15b7-43de-9b83-66f26435f31f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:02.388781 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-22b86471-15b7-43de-9b83-66f26435f31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:02.445076 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b86471-15b7-43de-9b83-66f26435f31f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:02.446560 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98420a4c-5537-436b-9865-e7fd956ff932 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:02.448769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a447710-658a-4956-b3a6-ae40bf06eec4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:02.449382 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a447710-658a-4956-b3a6-ae40bf06eec4 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:02.449662 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a447710-658a-4956-b3a6-ae40bf06eec4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:02.449936 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a447710-658a-4956-b3a6-ae40bf06eec4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:02.451108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de4fcf-d076-4d72-8b89-a94d623967bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:02.451747 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de4fcf-d076-4d72-8b89-a94d623967bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:02.452007 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de4fcf-d076-4d72-8b89-a94d623967bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:02.452275 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de4fcf-d076-4d72-8b89-a94d623967bf admin admin] PATH_INFO: `/v3/users/feedcd9bb713407c89d0ee9955c6eb38` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:02.453377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de4fcf-d076-4d72-8b89-a94d623967bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=feedcd9bb713407c89d0ee9955c6eb38)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:02.455323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de4fcf-d076-4d72-8b89-a94d623967bf admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:02.459469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a447710-658a-4956-b3a6-ae40bf06eec4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:02.460897 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a447710-658a-4956-b3a6-ae40bf06eec4 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:02.471977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1087/2189] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:32:02.508111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-07de4fcf-d076-4d72-8b89-a94d623967bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feedcd9bb713407c89d0ee9955c6eb38'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:02.515244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-07de4fcf-d076-4d72-8b89-a94d623967bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feedcd9bb713407c89d0ee9955c6eb38'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:02.528602 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-07de4fcf-d076-4d72-8b89-a94d623967bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feedcd9bb713407c89d0ee9955c6eb38'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:02.537403 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-07de4fcf-d076-4d72-8b89-a94d623967bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feedcd9bb713407c89d0ee9955c6eb38'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:02.538134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-07de4fcf-d076-4d72-8b89-a94d623967bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feedcd9bb713407c89d0ee9955c6eb38'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:02.544043 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1103/2190] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:02 2022] DELETE /identity/v3/users/feedcd9bb713407c89d0ee9955c6eb38 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:02.547258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a447710-658a-4956-b3a6-ae40bf06eec4 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:02.548136 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d872109-a14f-4ca5-9cae-5eb6753250c5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:02.548420 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d872109-a14f-4ca5-9cae-5eb6753250c5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:02.548692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d872109-a14f-4ca5-9cae-5eb6753250c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:02.556017 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a447710-658a-4956-b3a6-ae40bf06eec4 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:02.568723 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9d872109-a14f-4ca5-9cae-5eb6753250c5 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:02.570041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9d872109-a14f-4ca5-9cae-5eb6753250c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:02.594897 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:02.606454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd650c-cd54-41c0-a9bb-b9f822ab262b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:02.607370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd650c-cd54-41c0-a9bb-b9f822ab262b service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:02.607650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd650c-cd54-41c0-a9bb-b9f822ab262b service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:02.607922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd650c-cd54-41c0-a9bb-b9f822ab262b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:02.609693 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bebd650c-cd54-41c0-a9bb-b9f822ab262b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:02.633868 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1104/2191] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:32:02.643358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:02.644234 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d872109-a14f-4ca5-9cae-5eb6753250c5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:02.646021 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9d872109-a14f-4ca5-9cae-5eb6753250c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:02.674096 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd650c-cd54-41c0-a9bb-b9f822ab262b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:02.676266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd650c-cd54-41c0-a9bb-b9f822ab262b service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:02.686352 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1088/2192] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:02.705358 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d872109-a14f-4ca5-9cae-5eb6753250c5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:02.709014 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869802cc-b849-4f20-8e38-c924176dcc2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:02.709621 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869802cc-b849-4f20-8e38-c924176dcc2e service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:02.709893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869802cc-b849-4f20-8e38-c924176dcc2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:02.710189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869802cc-b849-4f20-8e38-c924176dcc2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:02.719609 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869802cc-b849-4f20-8e38-c924176dcc2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:02.721678 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869802cc-b849-4f20-8e38-c924176dcc2e service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:02.738611 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1105/2193] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:32:03.035404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd650c-cd54-41c0-a9bb-b9f822ab262b service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:03.036904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bebd650c-cd54-41c0-a9bb-b9f822ab262b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:03.054086 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd650c-cd54-41c0-a9bb-b9f822ab262b service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:03.059731 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f7fc3-084a-4c89-a379-983fc3c57ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:03.060382 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f7fc3-084a-4c89-a379-983fc3c57ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:03.060658 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f7fc3-084a-4c89-a379-983fc3c57ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:03.060930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f7fc3-084a-4c89-a379-983fc3c57ce4 admin admin] PATH_INFO: `/v3/projects/1147d14fb7b5456382c0097ecd3f66b4` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:03.062090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f7fc3-084a-4c89-a379-983fc3c57ce4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1147d14fb7b5456382c0097ecd3f66b4)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:03.064393 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f7fc3-084a-4c89-a379-983fc3c57ce4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:03.111847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-767f7fc3-084a-4c89-a379-983fc3c57ce4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1147d14fb7b5456382c0097ecd3f66b4', 'request_id': 'req-767f7fc3-084a-4c89-a379-983fc3c57ce4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:03.117781 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-767f7fc3-084a-4c89-a379-983fc3c57ce4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1147d14fb7b5456382c0097ecd3f66b4', 'request_id': 'req-767f7fc3-084a-4c89-a379-983fc3c57ce4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:03.123852 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-767f7fc3-084a-4c89-a379-983fc3c57ce4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1147d14fb7b5456382c0097ecd3f66b4', 'request_id': 'req-767f7fc3-084a-4c89-a379-983fc3c57ce4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:03.136182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-767f7fc3-084a-4c89-a379-983fc3c57ce4 admin admin] The token cache is being invalidate because project 1147d14fb7b5456382c0097ecd3f66b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:03.136474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-767f7fc3-084a-4c89-a379-983fc3c57ce4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:03.137115 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-767f7fc3-084a-4c89-a379-983fc3c57ce4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:03.138716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1089/2194] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:03 2022] DELETE /identity/v3/projects/1147d14fb7b5456382c0097ecd3f66b4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:05.450977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869802cc-b849-4f20-8e38-c924176dcc2e service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:05.452484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-869802cc-b849-4f20-8e38-c924176dcc2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:05.465322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869802cc-b849-4f20-8e38-c924176dcc2e service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:05.520508 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e1552-3e9f-496d-8b18-f579e51d57cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:05.521136 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e1552-3e9f-496d-8b18-f579e51d57cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:05.521417 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e1552-3e9f-496d-8b18-f579e51d57cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:05.521697 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e1552-3e9f-496d-8b18-f579e51d57cb admin admin] PATH_INFO: `/v3/users/fd3ee784609540cbba19adb3adb38bab` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:05.522822 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e1552-3e9f-496d-8b18-f579e51d57cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd3ee784609540cbba19adb3adb38bab)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:05.524799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e1552-3e9f-496d-8b18-f579e51d57cb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:05.570467 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4c7e1552-3e9f-496d-8b18-f579e51d57cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd3ee784609540cbba19adb3adb38bab'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:05.577697 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4c7e1552-3e9f-496d-8b18-f579e51d57cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd3ee784609540cbba19adb3adb38bab'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:05.589883 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4c7e1552-3e9f-496d-8b18-f579e51d57cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd3ee784609540cbba19adb3adb38bab'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:05.591846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:05.598216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4c7e1552-3e9f-496d-8b18-f579e51d57cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd3ee784609540cbba19adb3adb38bab'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:05.599157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4c7e1552-3e9f-496d-8b18-f579e51d57cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd3ee784609540cbba19adb3adb38bab'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:05.605661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1106/2195] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:05 2022] DELETE /identity/v3/users/fd3ee784609540cbba19adb3adb38bab => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:05.610372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc32cc1-49db-4295-9d52-b3e5444c16c3 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:05.610834 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc32cc1-49db-4295-9d52-b3e5444c16c3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:05.610949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc32cc1-49db-4295-9d52-b3e5444c16c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:05.630392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-0fc32cc1-49db-4295-9d52-b3e5444c16c3 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:05.631619 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0fc32cc1-49db-4295-9d52-b3e5444c16c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:05.689350 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1090/2196] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:05.699036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e1552-3e9f-496d-8b18-f579e51d57cb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:05.700464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4c7e1552-3e9f-496d-8b18-f579e51d57cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:05.758441 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e1552-3e9f-496d-8b18-f579e51d57cb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:05.762106 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297699c-779b-450f-8032-3d4211303faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:05.762708 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297699c-779b-450f-8032-3d4211303faa service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:05.762985 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297699c-779b-450f-8032-3d4211303faa service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:05.763299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297699c-779b-450f-8032-3d4211303faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:05.772191 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297699c-779b-450f-8032-3d4211303faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:05.773626 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297699c-779b-450f-8032-3d4211303faa service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:05.782850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1107/2197] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:06.046641 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc32cc1-49db-4295-9d52-b3e5444c16c3 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:06.048110 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0fc32cc1-49db-4295-9d52-b3e5444c16c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:06.060307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc32cc1-49db-4295-9d52-b3e5444c16c3 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:06.065021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:06.065664 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:06.065945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:06.066219 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] PATH_INFO: `/v3/projects/1702fd082d2c42cda1f04b2f1299b76f` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:06.067451 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] RBAC: Authorizing `identity:delete_project(project_id=1702fd082d2c42cda1f04b2f1299b76f)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:06.069418 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:06.120851 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1702fd082d2c42cda1f04b2f1299b76f', 'request_id': 'req-570f8ef5-08f2-47c4-acdc-ce10cd2892df'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.122077 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:06.127891 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1702fd082d2c42cda1f04b2f1299b76f', 'request_id': 'req-570f8ef5-08f2-47c4-acdc-ce10cd2892df'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.135812 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1702fd082d2c42cda1f04b2f1299b76f', 'request_id': 'req-570f8ef5-08f2-47c4-acdc-ce10cd2892df'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.149804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] The token cache is being invalidate because project 1702fd082d2c42cda1f04b2f1299b76f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:06.149804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.150444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.152009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1091/2198] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:06 2022] DELETE /identity/v3/projects/1702fd082d2c42cda1f04b2f1299b76f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:06.156248 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297699c-779b-450f-8032-3d4211303faa service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:06.166300 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297699c-779b-450f-8032-3d4211303faa service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:06.225651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb114d-bfde-4c62-a463-30944f395813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:06.226428 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb114d-bfde-4c62-a463-30944f395813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:06.226705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb114d-bfde-4c62-a463-30944f395813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:06.226976 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb114d-bfde-4c62-a463-30944f395813 admin admin] PATH_INFO: `/v3/users/0622e87436514c02a5726afe30011973` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:06.228117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb114d-bfde-4c62-a463-30944f395813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0622e87436514c02a5726afe30011973)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:06.230342 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb114d-bfde-4c62-a463-30944f395813 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:06.278650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-18cb114d-bfde-4c62-a463-30944f395813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0622e87436514c02a5726afe30011973'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.285501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-18cb114d-bfde-4c62-a463-30944f395813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0622e87436514c02a5726afe30011973'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.297237 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-18cb114d-bfde-4c62-a463-30944f395813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0622e87436514c02a5726afe30011973'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.304084 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-18cb114d-bfde-4c62-a463-30944f395813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0622e87436514c02a5726afe30011973'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.304714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-18cb114d-bfde-4c62-a463-30944f395813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0622e87436514c02a5726afe30011973'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.310527 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1108/2199] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:06 2022] DELETE /identity/v3/users/0622e87436514c02a5726afe30011973 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:06.581217 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:06.582706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:06.596309 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f8ef5-08f2-47c4-acdc-ce10cd2892df admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:06.659956 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eff32c-9502-42f1-bfbb-18dbb364d15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:06.660694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff32c-9502-42f1-bfbb-18dbb364d15e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:06.660824 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff32c-9502-42f1-bfbb-18dbb364d15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:06.661014 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff32c-9502-42f1-bfbb-18dbb364d15e admin admin] PATH_INFO: `/v3/projects/3043378b75e94a0f88fbe3dd06970466` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:06.662123 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eff32c-9502-42f1-bfbb-18dbb364d15e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3043378b75e94a0f88fbe3dd06970466)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:06.664085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eff32c-9502-42f1-bfbb-18dbb364d15e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:06.711387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-25eff32c-9502-42f1-bfbb-18dbb364d15e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3043378b75e94a0f88fbe3dd06970466', 'request_id': 'req-25eff32c-9502-42f1-bfbb-18dbb364d15e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.716817 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-25eff32c-9502-42f1-bfbb-18dbb364d15e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3043378b75e94a0f88fbe3dd06970466', 'request_id': 'req-25eff32c-9502-42f1-bfbb-18dbb364d15e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.722979 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-25eff32c-9502-42f1-bfbb-18dbb364d15e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3043378b75e94a0f88fbe3dd06970466', 'request_id': 'req-25eff32c-9502-42f1-bfbb-18dbb364d15e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.736247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-25eff32c-9502-42f1-bfbb-18dbb364d15e admin admin] The token cache is being invalidate because project 3043378b75e94a0f88fbe3dd06970466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:06.737060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-25eff32c-9502-42f1-bfbb-18dbb364d15e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.740288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-25eff32c-9502-42f1-bfbb-18dbb364d15e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:06.742039 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1092/2200] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:06 2022] DELETE /identity/v3/projects/3043378b75e94a0f88fbe3dd06970466 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:06.746702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53563058-5295-421a-bf51-70427235c591 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:06.747170 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53563058-5295-421a-bf51-70427235c591 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:06.747326 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53563058-5295-421a-bf51-70427235c591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:06.768000 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-53563058-5295-421a-bf51-70427235c591 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:06.769278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-53563058-5295-421a-bf51-70427235c591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:06.778204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:06.833732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1109/2201] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:06.841453 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb02e2-a6f0-4407-9235-c45a64ba0d9d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:06.841728 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb02e2-a6f0-4407-9235-c45a64ba0d9d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:06.842002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb02e2-a6f0-4407-9235-c45a64ba0d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:06.862491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-85fb02e2-a6f0-4407-9235-c45a64ba0d9d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:06.863757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-85fb02e2-a6f0-4407-9235-c45a64ba0d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:06.871516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1093/2202] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:06.875890 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53563058-5295-421a-bf51-70427235c591 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:06.887106 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53563058-5295-421a-bf51-70427235c591 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:06.891722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea00c2-169a-424c-9fe5-8a4bbbb44729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:06.892316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea00c2-169a-424c-9fe5-8a4bbbb44729 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:06.892595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea00c2-169a-424c-9fe5-8a4bbbb44729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:06.893004 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea00c2-169a-424c-9fe5-8a4bbbb44729 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:06.893781 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea00c2-169a-424c-9fe5-8a4bbbb44729 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:32:06.894088 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea00c2-169a-424c-9fe5-8a4bbbb44729 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:06.895787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea00c2-169a-424c-9fe5-8a4bbbb44729 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:06.908027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1110/2203] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:32:06.912876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb02e2-a6f0-4407-9235-c45a64ba0d9d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:06.922272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb02e2-a6f0-4407-9235-c45a64ba0d9d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:06.927630 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059cf89-95de-4630-87b5-c0f0459a4b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:06.928258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059cf89-95de-4630-87b5-c0f0459a4b11 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:06.928583 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059cf89-95de-4630-87b5-c0f0459a4b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:06.928841 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059cf89-95de-4630-87b5-c0f0459a4b11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:06.929495 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059cf89-95de-4630-87b5-c0f0459a4b11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:06.931536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059cf89-95de-4630-87b5-c0f0459a4b11 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:06.959750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1094/2204] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:06 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:06.963906 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea00c2-169a-424c-9fe5-8a4bbbb44729 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:06.974481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea00c2-169a-424c-9fe5-8a4bbbb44729 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:07.032945 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172aa728-37bc-4700-8366-3208f7e467ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:07.034059 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172aa728-37bc-4700-8366-3208f7e467ea admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:07.034336 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172aa728-37bc-4700-8366-3208f7e467ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:07.034608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172aa728-37bc-4700-8366-3208f7e467ea admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:07.035257 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172aa728-37bc-4700-8366-3208f7e467ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:07.037169 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172aa728-37bc-4700-8366-3208f7e467ea admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:07.058758 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1111/2205] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:06 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:07.063145 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059cf89-95de-4630-87b5-c0f0459a4b11 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:07.074195 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059cf89-95de-4630-87b5-c0f0459a4b11 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:07.078885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914b2a0-43a2-4315-9ba7-b9a1b8bbd298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:07.079520 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914b2a0-43a2-4315-9ba7-b9a1b8bbd298 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:07.079796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914b2a0-43a2-4315-9ba7-b9a1b8bbd298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:07.080065 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914b2a0-43a2-4315-9ba7-b9a1b8bbd298 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:07.081046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914b2a0-43a2-4315-9ba7-b9a1b8bbd298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:07.082641 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914b2a0-43a2-4315-9ba7-b9a1b8bbd298 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:07.093414 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1095/2206] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:07.098000 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172aa728-37bc-4700-8366-3208f7e467ea admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:07.108216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172aa728-37bc-4700-8366-3208f7e467ea admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:07.113896 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b540d92-c15a-4332-87ae-f6eb0fdb91e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:07.114491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b540d92-c15a-4332-87ae-f6eb0fdb91e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:07.114799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b540d92-c15a-4332-87ae-f6eb0fdb91e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:07.115097 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b540d92-c15a-4332-87ae-f6eb0fdb91e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:07.115741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b540d92-c15a-4332-87ae-f6eb0fdb91e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:07.117386 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b540d92-c15a-4332-87ae-f6eb0fdb91e5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:07.127951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1112/2207] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:07.132252 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914b2a0-43a2-4315-9ba7-b9a1b8bbd298 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:07.142903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914b2a0-43a2-4315-9ba7-b9a1b8bbd298 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:07.148221 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25119401-1653-4dd6-b3ba-e467a8ea96ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:07.148958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25119401-1653-4dd6-b3ba-e467a8ea96ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:07.149281 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25119401-1653-4dd6-b3ba-e467a8ea96ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:07.149589 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25119401-1653-4dd6-b3ba-e467a8ea96ef admin admin] PATH_INFO: `/v3/projects/43b7f17ff5e34ef990d5e5c2898da2e5/users/762f63cb23d74b4ea1d2fb8d6856ef3f/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:07.165657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25119401-1653-4dd6-b3ba-e467a8ea96ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=43b7f17ff5e34ef990d5e5c2898da2e5, user_id=762f63cb23d74b4ea1d2fb8d6856ef3f, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:07.168274 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25119401-1653-4dd6-b3ba-e467a8ea96ef admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:07.182252 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1096/2208] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:07 2022] PUT /identity/v3/projects/43b7f17ff5e34ef990d5e5c2898da2e5/users/762f63cb23d74b4ea1d2fb8d6856ef3f/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:07.196142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5240e4-5d4d-4893-b845-17c625bd9067 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:07.196433 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5240e4-5d4d-4893-b845-17c625bd9067 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:07.196746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5240e4-5d4d-4893-b845-17c625bd9067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:07.231668 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3f5240e4-5d4d-4893-b845-17c625bd9067 None None] MFA Rules not processed for user `762f63cb23d74b4ea1d2fb8d6856ef3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:07.233433 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3f5240e4-5d4d-4893-b845-17c625bd9067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:07.305297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1113/2209] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:07.310003 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25119401-1653-4dd6-b3ba-e467a8ea96ef admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:07.315994 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:07.378611 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25119401-1653-4dd6-b3ba-e467a8ea96ef admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:07.383403 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b8a26-511d-49ed-8469-31d086616125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:07.384002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b8a26-511d-49ed-8469-31d086616125 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:07.384303 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b8a26-511d-49ed-8469-31d086616125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:07.384572 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b8a26-511d-49ed-8469-31d086616125 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:07.385350 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b8a26-511d-49ed-8469-31d086616125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:07.387255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b8a26-511d-49ed-8469-31d086616125 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:07.415445 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1097/2210] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:07 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:07.420152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5240e4-5d4d-4893-b845-17c625bd9067 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:07.431434 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5240e4-5d4d-4893-b845-17c625bd9067 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:07.435994 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:07.489950 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298ca79-44e6-4c67-9e9e-7d1e6a8442b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:07.490581 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298ca79-44e6-4c67-9e9e-7d1e6a8442b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:07.490860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298ca79-44e6-4c67-9e9e-7d1e6a8442b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:07.491163 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298ca79-44e6-4c67-9e9e-7d1e6a8442b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:07.491804 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298ca79-44e6-4c67-9e9e-7d1e6a8442b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:07.493778 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298ca79-44e6-4c67-9e9e-7d1e6a8442b7 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:07.514582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1114/2211] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:07 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:07.519615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b8a26-511d-49ed-8469-31d086616125 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:07.530655 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b8a26-511d-49ed-8469-31d086616125 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:07.537273 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fa1f2-b8ca-4d20-98ce-43a43025ac9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:07.537965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fa1f2-b8ca-4d20-98ce-43a43025ac9e admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:07.538304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fa1f2-b8ca-4d20-98ce-43a43025ac9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:07.538707 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fa1f2-b8ca-4d20-98ce-43a43025ac9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:07.539547 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fa1f2-b8ca-4d20-98ce-43a43025ac9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:07.541431 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fa1f2-b8ca-4d20-98ce-43a43025ac9e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:07.554718 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1098/2212] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:07.558754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298ca79-44e6-4c67-9e9e-7d1e6a8442b7 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:07.569480 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298ca79-44e6-4c67-9e9e-7d1e6a8442b7 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:07.574069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fbf16-f09f-4f7c-9fd5-53954c2843f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:07.574715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fbf16-f09f-4f7c-9fd5-53954c2843f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:07.574992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fbf16-f09f-4f7c-9fd5-53954c2843f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:07.575299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fbf16-f09f-4f7c-9fd5-53954c2843f2 admin admin] PATH_INFO: `/v3/projects/c6eeae38b102408ea605933509dd4b81/users/c460ab34460049618a30622f827e077e/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:07.589969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fbf16-f09f-4f7c-9fd5-53954c2843f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6eeae38b102408ea605933509dd4b81, user_id=c460ab34460049618a30622f827e077e, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:07.592653 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fbf16-f09f-4f7c-9fd5-53954c2843f2 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:07.605181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1115/2213] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:07 2022] PUT /identity/v3/projects/c6eeae38b102408ea605933509dd4b81/users/c460ab34460049618a30622f827e077e/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:07.609505 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fa1f2-b8ca-4d20-98ce-43a43025ac9e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:07.620585 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fa1f2-b8ca-4d20-98ce-43a43025ac9e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:07.680088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126c22e-b854-4768-aadf-bc1d5a9b258a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:07.680826 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126c22e-b854-4768-aadf-bc1d5a9b258a admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:07.680938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126c22e-b854-4768-aadf-bc1d5a9b258a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:07.681186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126c22e-b854-4768-aadf-bc1d5a9b258a admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:07.681904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126c22e-b854-4768-aadf-bc1d5a9b258a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:07.683470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126c22e-b854-4768-aadf-bc1d5a9b258a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:07.693878 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1099/2214] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:07.698144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fbf16-f09f-4f7c-9fd5-53954c2843f2 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:07.709566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fbf16-f09f-4f7c-9fd5-53954c2843f2 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:07.714314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa6461-498c-4c32-9001-322bf1041085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:07.714964 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa6461-498c-4c32-9001-322bf1041085 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:07.715306 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa6461-498c-4c32-9001-322bf1041085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:07.715551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa6461-498c-4c32-9001-322bf1041085 admin admin] PATH_INFO: `/v3/projects/c6eeae38b102408ea605933509dd4b81/users/c460ab34460049618a30622f827e077e/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:07.717490 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa6461-498c-4c32-9001-322bf1041085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6eeae38b102408ea605933509dd4b81, user_id=c460ab34460049618a30622f827e077e, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:07.720167 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa6461-498c-4c32-9001-322bf1041085 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:07.731194 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1116/2215] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:07 2022] PUT /identity/v3/projects/c6eeae38b102408ea605933509dd4b81/users/c460ab34460049618a30622f827e077e/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:07.741575 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22625048-71f7-435b-9826-281e1eaea71e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:07.741864 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22625048-71f7-435b-9826-281e1eaea71e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:07.742143 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22625048-71f7-435b-9826-281e1eaea71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:07.771729 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-22625048-71f7-435b-9826-281e1eaea71e None None] MFA Rules not processed for user `c460ab34460049618a30622f827e077e`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:07.772949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-22625048-71f7-435b-9826-281e1eaea71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:07.842930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:07.850874 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1100/2216] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:07.858252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa6461-498c-4c32-9001-322bf1041085 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:07.860024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a2aa6461-498c-4c32-9001-322bf1041085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:07.918543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa6461-498c-4c32-9001-322bf1041085 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:07.922177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce21d6-8e74-4f4f-88a4-ff6033c35c40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:07.922783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce21d6-8e74-4f4f-88a4-ff6033c35c40 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:07.923125 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce21d6-8e74-4f4f-88a4-ff6033c35c40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:07.923382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce21d6-8e74-4f4f-88a4-ff6033c35c40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:07.932601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce21d6-8e74-4f4f-88a4-ff6033c35c40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:07.933997 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce21d6-8e74-4f4f-88a4-ff6033c35c40 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:07.943819 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1117/2217] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:09.386184 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22625048-71f7-435b-9826-281e1eaea71e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:09.452796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22625048-71f7-435b-9826-281e1eaea71e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:09.457781 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d3c70-23c7-4c7d-a853-2d057bcaaeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:09.458417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d3c70-23c7-4c7d-a853-2d057bcaaeaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:09.458694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d3c70-23c7-4c7d-a853-2d057bcaaeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:09.458970 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d3c70-23c7-4c7d-a853-2d057bcaaeaa admin admin] PATH_INFO: `/v3/users/762f63cb23d74b4ea1d2fb8d6856ef3f` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:09.460118 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d3c70-23c7-4c7d-a853-2d057bcaaeaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=762f63cb23d74b4ea1d2fb8d6856ef3f)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:09.462034 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d3c70-23c7-4c7d-a853-2d057bcaaeaa admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:09.508912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e63d3c70-23c7-4c7d-a853-2d057bcaaeaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '762f63cb23d74b4ea1d2fb8d6856ef3f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:09.516304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e63d3c70-23c7-4c7d-a853-2d057bcaaeaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '762f63cb23d74b4ea1d2fb8d6856ef3f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:09.516962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e63d3c70-23c7-4c7d-a853-2d057bcaaeaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '762f63cb23d74b4ea1d2fb8d6856ef3f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:09.529250 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e63d3c70-23c7-4c7d-a853-2d057bcaaeaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '762f63cb23d74b4ea1d2fb8d6856ef3f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:09.537155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e63d3c70-23c7-4c7d-a853-2d057bcaaeaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '762f63cb23d74b4ea1d2fb8d6856ef3f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:09.543207 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1101/2218] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:09 2022] DELETE /identity/v3/users/762f63cb23d74b4ea1d2fb8d6856ef3f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:09.547651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d489d5-e98a-49b8-8098-fc3f2adec32f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:09.548112 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d489d5-e98a-49b8-8098-fc3f2adec32f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:09.548229 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d489d5-e98a-49b8-8098-fc3f2adec32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:09.567029 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-88d489d5-e98a-49b8-8098-fc3f2adec32f None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:09.568219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-88d489d5-e98a-49b8-8098-fc3f2adec32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:09.626808 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1118/2219] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:09.636631 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d3c70-23c7-4c7d-a853-2d057bcaaeaa admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:09.638024 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e63d3c70-23c7-4c7d-a853-2d057bcaaeaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:09.698276 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d3c70-23c7-4c7d-a853-2d057bcaaeaa admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:09.701955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287e792-f9ed-4815-aeed-01e0530aabef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:09.702554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287e792-f9ed-4815-aeed-01e0530aabef service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:09.702828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287e792-f9ed-4815-aeed-01e0530aabef service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:09.703125 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287e792-f9ed-4815-aeed-01e0530aabef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:09.712387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287e792-f9ed-4815-aeed-01e0530aabef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:09.713798 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287e792-f9ed-4815-aeed-01e0530aabef service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:09.723811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1102/2220] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:10.010733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d489d5-e98a-49b8-8098-fc3f2adec32f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:10.012839 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-88d489d5-e98a-49b8-8098-fc3f2adec32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:10.022387 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:10.030269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d489d5-e98a-49b8-8098-fc3f2adec32f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:10.037078 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:10.038252 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:10.038927 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:10.039697 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] PATH_INFO: `/v3/projects/43b7f17ff5e34ef990d5e5c2898da2e5` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:10.040946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=43b7f17ff5e34ef990d5e5c2898da2e5)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:10.042928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:10.099201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43b7f17ff5e34ef990d5e5c2898da2e5', 'request_id': 'req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.105902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43b7f17ff5e34ef990d5e5c2898da2e5', 'request_id': 'req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.113523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43b7f17ff5e34ef990d5e5c2898da2e5', 'request_id': 'req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.126405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] The token cache is being invalidate because project 43b7f17ff5e34ef990d5e5c2898da2e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:10.126695 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.128025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.129550 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1119/2221] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:10 2022] DELETE /identity/v3/projects/43b7f17ff5e34ef990d5e5c2898da2e5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:10.133951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287e792-f9ed-4815-aeed-01e0530aabef service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:10.147096 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287e792-f9ed-4815-aeed-01e0530aabef service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:10.173466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:10.221235 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f9d2f-40ed-4aa2-ab59-4a512f76d6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:10.221947 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f9d2f-40ed-4aa2-ab59-4a512f76d6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:10.222250 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f9d2f-40ed-4aa2-ab59-4a512f76d6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:10.222520 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f9d2f-40ed-4aa2-ab59-4a512f76d6d3 admin admin] PATH_INFO: `/v3/users/c460ab34460049618a30622f827e077e` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:10.223711 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f9d2f-40ed-4aa2-ab59-4a512f76d6d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c460ab34460049618a30622f827e077e)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:10.225801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f9d2f-40ed-4aa2-ab59-4a512f76d6d3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:10.275573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0d5f9d2f-40ed-4aa2-ab59-4a512f76d6d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c460ab34460049618a30622f827e077e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.283711 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0d5f9d2f-40ed-4aa2-ab59-4a512f76d6d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c460ab34460049618a30622f827e077e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.284501 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0d5f9d2f-40ed-4aa2-ab59-4a512f76d6d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c460ab34460049618a30622f827e077e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.297561 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0d5f9d2f-40ed-4aa2-ab59-4a512f76d6d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c460ab34460049618a30622f827e077e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.304553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0d5f9d2f-40ed-4aa2-ab59-4a512f76d6d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c460ab34460049618a30622f827e077e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.310754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1103/2222] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:10 2022] DELETE /identity/v3/users/c460ab34460049618a30622f827e077e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:10.591279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:10.592731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:10.607150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ede87-6fd7-4cb2-b42a-9ecdb7dadcbf admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:10.679368 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddbbe2-15c4-4d35-b82f-46750923149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:10.680007 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddbbe2-15c4-4d35-b82f-46750923149a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:10.680286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddbbe2-15c4-4d35-b82f-46750923149a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:10.680579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddbbe2-15c4-4d35-b82f-46750923149a admin admin] PATH_INFO: `/v3/projects/c6eeae38b102408ea605933509dd4b81` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:10.681675 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbddbbe2-15c4-4d35-b82f-46750923149a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6eeae38b102408ea605933509dd4b81)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:10.683629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbddbbe2-15c4-4d35-b82f-46750923149a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:10.735225 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dbddbbe2-15c4-4d35-b82f-46750923149a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6eeae38b102408ea605933509dd4b81', 'request_id': 'req-dbddbbe2-15c4-4d35-b82f-46750923149a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.741149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dbddbbe2-15c4-4d35-b82f-46750923149a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6eeae38b102408ea605933509dd4b81', 'request_id': 'req-dbddbbe2-15c4-4d35-b82f-46750923149a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.747393 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dbddbbe2-15c4-4d35-b82f-46750923149a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6eeae38b102408ea605933509dd4b81', 'request_id': 'req-dbddbbe2-15c4-4d35-b82f-46750923149a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.760536 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dbddbbe2-15c4-4d35-b82f-46750923149a admin admin] The token cache is being invalidate because project c6eeae38b102408ea605933509dd4b81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:10.760825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dbddbbe2-15c4-4d35-b82f-46750923149a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.761480 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dbddbbe2-15c4-4d35-b82f-46750923149a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:10.763028 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1120/2223] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:10 2022] DELETE /identity/v3/projects/c6eeae38b102408ea605933509dd4b81 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:10.768621 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67175075-04db-4fcd-ada0-04e6d02ce263 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:10.769201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67175075-04db-4fcd-ada0-04e6d02ce263 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:10.769201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67175075-04db-4fcd-ada0-04e6d02ce263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:10.788721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-67175075-04db-4fcd-ada0-04e6d02ce263 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:10.790067 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-67175075-04db-4fcd-ada0-04e6d02ce263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:10.850422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1104/2224] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:10.859444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd4686-fac3-49f6-991b-8fd25fde2819 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:10.859733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd4686-fac3-49f6-991b-8fd25fde2819 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:10.860025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd4686-fac3-49f6-991b-8fd25fde2819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:10.882118 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-dbdd4686-fac3-49f6-991b-8fd25fde2819 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:10.882891 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-dbdd4686-fac3-49f6-991b-8fd25fde2819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:10.891237 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1121/2225] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:10.896102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67175075-04db-4fcd-ada0-04e6d02ce263 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:10.906240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67175075-04db-4fcd-ada0-04e6d02ce263 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:10.911305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5459f1-a20b-4590-b974-08716bacbda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:10.911941 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5459f1-a20b-4590-b974-08716bacbda4 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:10.912238 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5459f1-a20b-4590-b974-08716bacbda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:10.912531 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5459f1-a20b-4590-b974-08716bacbda4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:10.913222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5459f1-a20b-4590-b974-08716bacbda4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:32:10.913548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5459f1-a20b-4590-b974-08716bacbda4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:10.915234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5459f1-a20b-4590-b974-08716bacbda4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:10.926866 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1105/2226] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:32:10.931157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd4686-fac3-49f6-991b-8fd25fde2819 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:10.941503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd4686-fac3-49f6-991b-8fd25fde2819 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:10.946302 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a8303-f8c3-493d-baae-bc0a65ad562e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:10.946895 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a8303-f8c3-493d-baae-bc0a65ad562e admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:10.947216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a8303-f8c3-493d-baae-bc0a65ad562e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:10.947495 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a8303-f8c3-493d-baae-bc0a65ad562e admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:10.948124 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a8303-f8c3-493d-baae-bc0a65ad562e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:10.950050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a8303-f8c3-493d-baae-bc0a65ad562e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:10.960608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:10.978697 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1122/2227] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:10 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:10.983570 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5459f1-a20b-4590-b974-08716bacbda4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:10.995684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5459f1-a20b-4590-b974-08716bacbda4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:11.024037 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:11.055892 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41f1ea-b6fe-46aa-ba8f-49ccef984139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:11.056501 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41f1ea-b6fe-46aa-ba8f-49ccef984139 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:11.056781 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41f1ea-b6fe-46aa-ba8f-49ccef984139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:11.057057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41f1ea-b6fe-46aa-ba8f-49ccef984139 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:11.057692 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41f1ea-b6fe-46aa-ba8f-49ccef984139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:11.059656 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41f1ea-b6fe-46aa-ba8f-49ccef984139 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:11.082309 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1106/2228] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:10 2022] POST /identity/v3/users => generated 301 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:11.086720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a8303-f8c3-493d-baae-bc0a65ad562e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:11.097090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a8303-f8c3-493d-baae-bc0a65ad562e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:11.101714 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a0274-bede-4f53-8a2b-cde95e11f448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:11.102304 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a0274-bede-4f53-8a2b-cde95e11f448 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:11.102559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a0274-bede-4f53-8a2b-cde95e11f448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:11.102825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a0274-bede-4f53-8a2b-cde95e11f448 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:11.103526 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a0274-bede-4f53-8a2b-cde95e11f448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:11.105068 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a0274-bede-4f53-8a2b-cde95e11f448 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:11.116145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1123/2229] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:32:11.120136 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41f1ea-b6fe-46aa-ba8f-49ccef984139 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:11.131253 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41f1ea-b6fe-46aa-ba8f-49ccef984139 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:11.137157 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c99281-418d-43a5-a433-bb0d4bed5a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:11.137867 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c99281-418d-43a5-a433-bb0d4bed5a3b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:11.138229 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c99281-418d-43a5-a433-bb0d4bed5a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:11.138539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c99281-418d-43a5-a433-bb0d4bed5a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:11.139294 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c99281-418d-43a5-a433-bb0d4bed5a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:11.141211 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c99281-418d-43a5-a433-bb0d4bed5a3b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:11.152994 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1107/2230] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:11.157121 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a0274-bede-4f53-8a2b-cde95e11f448 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:11.167122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a0274-bede-4f53-8a2b-cde95e11f448 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:11.171975 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59f162-6ab0-4795-a009-08c92e82d807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:11.172647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59f162-6ab0-4795-a009-08c92e82d807 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:11.172950 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59f162-6ab0-4795-a009-08c92e82d807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:11.173229 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59f162-6ab0-4795-a009-08c92e82d807 admin admin] PATH_INFO: `/v3/projects/01bf52316c5548459bd53b440ef79878/users/fc6bd40b152a45ef87b933a904c2ff22/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:11.188693 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59f162-6ab0-4795-a009-08c92e82d807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01bf52316c5548459bd53b440ef79878, user_id=fc6bd40b152a45ef87b933a904c2ff22, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:11.192177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59f162-6ab0-4795-a009-08c92e82d807 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:11.204931 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1124/2231] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:11 2022] PUT /identity/v3/projects/01bf52316c5548459bd53b440ef79878/users/fc6bd40b152a45ef87b933a904c2ff22/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:11.215914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d6977-17d1-4b63-b13f-a33db470c3fb None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:11.216205 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d6977-17d1-4b63-b13f-a33db470c3fb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:11.216565 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d6977-17d1-4b63-b13f-a33db470c3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:11.246106 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-189d6977-17d1-4b63-b13f-a33db470c3fb None None] MFA Rules not processed for user `fc6bd40b152a45ef87b933a904c2ff22`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:11.247344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-189d6977-17d1-4b63-b13f-a33db470c3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:11.313307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1108/2232] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:11 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:11.318323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59f162-6ab0-4795-a009-08c92e82d807 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:11.381486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59f162-6ab0-4795-a009-08c92e82d807 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:11.386047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ceb95-8bf5-458d-b898-9a8fb05ddaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:11.386685 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ceb95-8bf5-458d-b898-9a8fb05ddaa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:11.386964 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ceb95-8bf5-458d-b898-9a8fb05ddaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:11.387268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ceb95-8bf5-458d-b898-9a8fb05ddaa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:11.387909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ceb95-8bf5-458d-b898-9a8fb05ddaa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:11.389818 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ceb95-8bf5-458d-b898-9a8fb05ddaa6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:11.417134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1125/2233] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:11 2022] POST /identity/v3/projects => generated 399 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:11.421272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d6977-17d1-4b63-b13f-a33db470c3fb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:11.432912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d6977-17d1-4b63-b13f-a33db470c3fb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:11.489985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad37b0a-7b6c-4d5d-90c0-6f70c54a90ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:11.490611 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad37b0a-7b6c-4d5d-90c0-6f70c54a90ac admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:11.490903 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad37b0a-7b6c-4d5d-90c0-6f70c54a90ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:11.491191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad37b0a-7b6c-4d5d-90c0-6f70c54a90ac admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:11.491838 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad37b0a-7b6c-4d5d-90c0-6f70c54a90ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:11.493792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad37b0a-7b6c-4d5d-90c0-6f70c54a90ac admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:11.513999 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1109/2234] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:11 2022] POST /identity/v3/users => generated 306 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:11.518758 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ceb95-8bf5-458d-b898-9a8fb05ddaa6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:11.531402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ceb95-8bf5-458d-b898-9a8fb05ddaa6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:11.537329 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cc2c6-6abe-42c7-97e5-03f079b2f19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:11.537923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cc2c6-6abe-42c7-97e5-03f079b2f19a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:11.538199 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cc2c6-6abe-42c7-97e5-03f079b2f19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:11.538466 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cc2c6-6abe-42c7-97e5-03f079b2f19a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:11.539138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876cc2c6-6abe-42c7-97e5-03f079b2f19a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:11.541332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876cc2c6-6abe-42c7-97e5-03f079b2f19a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:11.554400 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1126/2235] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:11.558685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad37b0a-7b6c-4d5d-90c0-6f70c54a90ac admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:11.568718 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad37b0a-7b6c-4d5d-90c0-6f70c54a90ac admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:11.573291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca1543-045c-49b2-b3e5-5e0ea9b4e99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:11.573932 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca1543-045c-49b2-b3e5-5e0ea9b4e99c admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:11.574208 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca1543-045c-49b2-b3e5-5e0ea9b4e99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:11.574477 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca1543-045c-49b2-b3e5-5e0ea9b4e99c admin admin] PATH_INFO: `/v3/projects/57146134daf34fa78c576bf0c970fcb1/users/b9be285533994c59b66e18c968e8ce25/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:11.589803 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca1543-045c-49b2-b3e5-5e0ea9b4e99c admin admin] RBAC: Authorizing `identity:create_grant(project_id=57146134daf34fa78c576bf0c970fcb1, user_id=b9be285533994c59b66e18c968e8ce25, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:11.592670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca1543-045c-49b2-b3e5-5e0ea9b4e99c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:11.606246 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1110/2236] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:11 2022] PUT /identity/v3/projects/57146134daf34fa78c576bf0c970fcb1/users/b9be285533994c59b66e18c968e8ce25/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:11.610438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cc2c6-6abe-42c7-97e5-03f079b2f19a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:11.622558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cc2c6-6abe-42c7-97e5-03f079b2f19a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:11.682993 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056813f-0134-49f4-8e5c-9674ad4c0c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:11.683715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056813f-0134-49f4-8e5c-9674ad4c0c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:11.683995 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056813f-0134-49f4-8e5c-9674ad4c0c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:11.684379 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056813f-0134-49f4-8e5c-9674ad4c0c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:11.685134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056813f-0134-49f4-8e5c-9674ad4c0c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:11.686973 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056813f-0134-49f4-8e5c-9674ad4c0c1c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:11.698644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1127/2237] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:11.703177 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca1543-045c-49b2-b3e5-5e0ea9b4e99c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:11.708972 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:11.715181 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca1543-045c-49b2-b3e5-5e0ea9b4e99c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:11.721142 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7e554-147b-41fc-945c-c6d726d1f898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:11.721882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7e554-147b-41fc-945c-c6d726d1f898 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:11.722160 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7e554-147b-41fc-945c-c6d726d1f898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:11.722464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7e554-147b-41fc-945c-c6d726d1f898 admin admin] PATH_INFO: `/v3/projects/57146134daf34fa78c576bf0c970fcb1/users/b9be285533994c59b66e18c968e8ce25/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:11.724423 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7e554-147b-41fc-945c-c6d726d1f898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57146134daf34fa78c576bf0c970fcb1, user_id=b9be285533994c59b66e18c968e8ce25, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:11.727163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7e554-147b-41fc-945c-c6d726d1f898 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:11.738671 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1111/2238] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:11 2022] PUT /identity/v3/projects/57146134daf34fa78c576bf0c970fcb1/users/b9be285533994c59b66e18c968e8ce25/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:11.749245 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2a5dd-92f4-43e7-a4b5-bf239c2c8a0d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:11.749607 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2a5dd-92f4-43e7-a4b5-bf239c2c8a0d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:11.749883 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2a5dd-92f4-43e7-a4b5-bf239c2c8a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:11.779250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a4f2a5dd-92f4-43e7-a4b5-bf239c2c8a0d None None] MFA Rules not processed for user `b9be285533994c59b66e18c968e8ce25`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:11.780530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a4f2a5dd-92f4-43e7-a4b5-bf239c2c8a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:11.793452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:11.852332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1128/2239] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:11 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:11.863878 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7e554-147b-41fc-945c-c6d726d1f898 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:11.865599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-70d7e554-147b-41fc-945c-c6d726d1f898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:11.925926 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7e554-147b-41fc-945c-c6d726d1f898 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:11.929598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f35b1b-717e-41fc-89b3-0e31d89a4feb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:11.930206 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f35b1b-717e-41fc-89b3-0e31d89a4feb service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:11.930484 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f35b1b-717e-41fc-89b3-0e31d89a4feb service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:11.930757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f35b1b-717e-41fc-89b3-0e31d89a4feb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:11.940220 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f35b1b-717e-41fc-89b3-0e31d89a4feb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:11.941579 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f35b1b-717e-41fc-89b3-0e31d89a4feb service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:11.950937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1112/2240] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:11 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:12.028839 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2a5dd-92f4-43e7-a4b5-bf239c2c8a0d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:12.038530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2a5dd-92f4-43e7-a4b5-bf239c2c8a0d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:12.042100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5721fb-5f19-49ee-9f58-79fa7d138c94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:12.042696 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5721fb-5f19-49ee-9f58-79fa7d138c94 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:12.042973 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5721fb-5f19-49ee-9f58-79fa7d138c94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:12.043272 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5721fb-5f19-49ee-9f58-79fa7d138c94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:12.098776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5721fb-5f19-49ee-9f58-79fa7d138c94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:12.100250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5721fb-5f19-49ee-9f58-79fa7d138c94 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:12.109283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1129/2241] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:12 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:32:16.256755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f35b1b-717e-41fc-89b3-0e31d89a4feb service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:16.322651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f35b1b-717e-41fc-89b3-0e31d89a4feb service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:16.327606 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88cd3b-3252-4614-aee8-8c7a1ea02cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:16.328242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88cd3b-3252-4614-aee8-8c7a1ea02cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:16.328522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88cd3b-3252-4614-aee8-8c7a1ea02cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:16.328825 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88cd3b-3252-4614-aee8-8c7a1ea02cfb admin admin] PATH_INFO: `/v3/users/fc6bd40b152a45ef87b933a904c2ff22` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:16.330440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88cd3b-3252-4614-aee8-8c7a1ea02cfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc6bd40b152a45ef87b933a904c2ff22)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:16.332596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88cd3b-3252-4614-aee8-8c7a1ea02cfb admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:16.380469 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ef88cd3b-3252-4614-aee8-8c7a1ea02cfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc6bd40b152a45ef87b933a904c2ff22'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:16.388302 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ef88cd3b-3252-4614-aee8-8c7a1ea02cfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc6bd40b152a45ef87b933a904c2ff22'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:16.388976 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ef88cd3b-3252-4614-aee8-8c7a1ea02cfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc6bd40b152a45ef87b933a904c2ff22'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:16.405351 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ef88cd3b-3252-4614-aee8-8c7a1ea02cfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc6bd40b152a45ef87b933a904c2ff22'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:16.415034 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ef88cd3b-3252-4614-aee8-8c7a1ea02cfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc6bd40b152a45ef87b933a904c2ff22'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:16.425742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1113/2242] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:16 2022] DELETE /identity/v3/users/fc6bd40b152a45ef87b933a904c2ff22 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:16.427520 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90d558-6d6e-4eb8-b469-eae3bf80fc01 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:16.427990 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90d558-6d6e-4eb8-b469-eae3bf80fc01 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:16.428105 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90d558-6d6e-4eb8-b469-eae3bf80fc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:16.452484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bc90d558-6d6e-4eb8-b469-eae3bf80fc01 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:16.453707 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bc90d558-6d6e-4eb8-b469-eae3bf80fc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:16.537521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1130/2243] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:16.544179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88cd3b-3252-4614-aee8-8c7a1ea02cfb admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:16.546186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ef88cd3b-3252-4614-aee8-8c7a1ea02cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:16.578698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:16.684973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88cd3b-3252-4614-aee8-8c7a1ea02cfb admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:16.694570 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25768606-7a91-4e31-9743-ce841312a5db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:16.695475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25768606-7a91-4e31-9743-ce841312a5db service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:16.695821 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25768606-7a91-4e31-9743-ce841312a5db service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:16.696161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25768606-7a91-4e31-9743-ce841312a5db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:16.714801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25768606-7a91-4e31-9743-ce841312a5db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:16.716472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25768606-7a91-4e31-9743-ce841312a5db service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:16.748941 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1114/2244] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:17.076161 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90d558-6d6e-4eb8-b469-eae3bf80fc01 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:17.077603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bc90d558-6d6e-4eb8-b469-eae3bf80fc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:17.090607 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90d558-6d6e-4eb8-b469-eae3bf80fc01 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:17.096545 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:17.097342 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:17.097617 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:17.098357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] PATH_INFO: `/v3/projects/01bf52316c5548459bd53b440ef79878` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:17.100009 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01bf52316c5548459bd53b440ef79878)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:17.101496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:17.153329 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:17.159900 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01bf52316c5548459bd53b440ef79878', 'request_id': 'req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.166172 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01bf52316c5548459bd53b440ef79878', 'request_id': 'req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.174423 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01bf52316c5548459bd53b440ef79878', 'request_id': 'req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.187956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] The token cache is being invalidate because project 01bf52316c5548459bd53b440ef79878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:17.188176 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.189230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.191152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1131/2245] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:17 2022] DELETE /identity/v3/projects/01bf52316c5548459bd53b440ef79878 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:17.195454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25768606-7a91-4e31-9743-ce841312a5db service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:17.207272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25768606-7a91-4e31-9743-ce841312a5db service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:17.265720 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad59a6-ce3d-44d4-a67b-4c6bb49d1b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:17.266420 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad59a6-ce3d-44d4-a67b-4c6bb49d1b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:17.266739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad59a6-ce3d-44d4-a67b-4c6bb49d1b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:17.267117 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad59a6-ce3d-44d4-a67b-4c6bb49d1b88 admin admin] PATH_INFO: `/v3/users/b9be285533994c59b66e18c968e8ce25` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:17.268265 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad59a6-ce3d-44d4-a67b-4c6bb49d1b88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9be285533994c59b66e18c968e8ce25)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:17.270196 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad59a6-ce3d-44d4-a67b-4c6bb49d1b88 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:17.319935 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ecad59a6-ce3d-44d4-a67b-4c6bb49d1b88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9be285533994c59b66e18c968e8ce25'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.328185 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ecad59a6-ce3d-44d4-a67b-4c6bb49d1b88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9be285533994c59b66e18c968e8ce25'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.329074 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ecad59a6-ce3d-44d4-a67b-4c6bb49d1b88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9be285533994c59b66e18c968e8ce25'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.342631 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ecad59a6-ce3d-44d4-a67b-4c6bb49d1b88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9be285533994c59b66e18c968e8ce25'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.350484 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ecad59a6-ce3d-44d4-a67b-4c6bb49d1b88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9be285533994c59b66e18c968e8ce25'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.357376 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1115/2246] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:17 2022] DELETE /identity/v3/users/b9be285533994c59b66e18c968e8ce25 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:17.672873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:17.675564 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:17.707571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdd8de-641a-40b8-84a1-b7927c9c5e55 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:17.786743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd206b-b029-46d5-b704-023b46162c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:17.787442 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd206b-b029-46d5-b704-023b46162c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:17.787823 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd206b-b029-46d5-b704-023b46162c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:17.788183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd206b-b029-46d5-b704-023b46162c95 admin admin] PATH_INFO: `/v3/projects/57146134daf34fa78c576bf0c970fcb1` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:17.789528 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd206b-b029-46d5-b704-023b46162c95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57146134daf34fa78c576bf0c970fcb1)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:17.791835 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd206b-b029-46d5-b704-023b46162c95 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:17.855971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bdfd206b-b029-46d5-b704-023b46162c95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57146134daf34fa78c576bf0c970fcb1', 'request_id': 'req-bdfd206b-b029-46d5-b704-023b46162c95'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.866069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bdfd206b-b029-46d5-b704-023b46162c95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57146134daf34fa78c576bf0c970fcb1', 'request_id': 'req-bdfd206b-b029-46d5-b704-023b46162c95'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.873292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bdfd206b-b029-46d5-b704-023b46162c95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57146134daf34fa78c576bf0c970fcb1', 'request_id': 'req-bdfd206b-b029-46d5-b704-023b46162c95'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.891574 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bdfd206b-b029-46d5-b704-023b46162c95 admin admin] The token cache is being invalidate because project 57146134daf34fa78c576bf0c970fcb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:17.891946 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bdfd206b-b029-46d5-b704-023b46162c95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.892831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bdfd206b-b029-46d5-b704-023b46162c95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:17.894699 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1132/2247] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:17 2022] DELETE /identity/v3/projects/57146134daf34fa78c576bf0c970fcb1 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:17.900240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b460c-3982-41dd-892e-39f1b693eb5d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:17.900673 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b460c-3982-41dd-892e-39f1b693eb5d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:17.901814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b460c-3982-41dd-892e-39f1b693eb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:17.929942 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-839b460c-3982-41dd-892e-39f1b693eb5d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:17.931425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-839b460c-3982-41dd-892e-39f1b693eb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:17.977009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:18.012666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1116/2248] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:18.023027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621dd98-6b55-401f-9862-3db40b430b50 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:18.023393 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621dd98-6b55-401f-9862-3db40b430b50 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:18.023607 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621dd98-6b55-401f-9862-3db40b430b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:18.045176 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6621dd98-6b55-401f-9862-3db40b430b50 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:18.046438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6621dd98-6b55-401f-9862-3db40b430b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:18.056785 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1133/2249] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:18.061285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b460c-3982-41dd-892e-39f1b693eb5d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:18.071355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b460c-3982-41dd-892e-39f1b693eb5d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:18.076291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224eb53-fa3e-4d91-a370-45e87811714f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:18.076917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224eb53-fa3e-4d91-a370-45e87811714f admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:18.077202 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224eb53-fa3e-4d91-a370-45e87811714f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:18.077491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224eb53-fa3e-4d91-a370-45e87811714f admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:18.078397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224eb53-fa3e-4d91-a370-45e87811714f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:32:18.078704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224eb53-fa3e-4d91-a370-45e87811714f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:18.080850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224eb53-fa3e-4d91-a370-45e87811714f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:18.094964 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1117/2250] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:32:18.100212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621dd98-6b55-401f-9862-3db40b430b50 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:18.112890 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621dd98-6b55-401f-9862-3db40b430b50 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:18.117605 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff113a-69ee-478b-9a77-72cb889714a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:18.118222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff113a-69ee-478b-9a77-72cb889714a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:18.118500 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff113a-69ee-478b-9a77-72cb889714a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:18.118771 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff113a-69ee-478b-9a77-72cb889714a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:18.120090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff113a-69ee-478b-9a77-72cb889714a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:18.122204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff113a-69ee-478b-9a77-72cb889714a9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:18.160629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1134/2251] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:18 2022] POST /identity/v3/projects => generated 407 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:18.161751 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224eb53-fa3e-4d91-a370-45e87811714f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:18.171944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224eb53-fa3e-4d91-a370-45e87811714f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:18.245327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13572e07-fc88-4043-a85d-fd7861aeaf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:18.245954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13572e07-fc88-4043-a85d-fd7861aeaf2f admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:18.249682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13572e07-fc88-4043-a85d-fd7861aeaf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:18.249974 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13572e07-fc88-4043-a85d-fd7861aeaf2f admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:18.250623 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13572e07-fc88-4043-a85d-fd7861aeaf2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:18.252980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13572e07-fc88-4043-a85d-fd7861aeaf2f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:18.278986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1118/2252] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:18 2022] POST /identity/v3/users => generated 304 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:18.285143 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff113a-69ee-478b-9a77-72cb889714a9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:18.321258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff113a-69ee-478b-9a77-72cb889714a9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:18.332335 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09569a2-4ede-4d53-b171-4567c549a20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:18.333144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09569a2-4ede-4d53-b171-4567c549a20a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:18.334498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09569a2-4ede-4d53-b171-4567c549a20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:18.334867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09569a2-4ede-4d53-b171-4567c549a20a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:18.337134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09569a2-4ede-4d53-b171-4567c549a20a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:18.339101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09569a2-4ede-4d53-b171-4567c549a20a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:18.351690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1135/2253] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:18.355666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13572e07-fc88-4043-a85d-fd7861aeaf2f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:18.369626 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13572e07-fc88-4043-a85d-fd7861aeaf2f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:18.375285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676a220-d43b-4fa1-8abb-c70aef51ba1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:18.375982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676a220-d43b-4fa1-8abb-c70aef51ba1b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:18.376845 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676a220-d43b-4fa1-8abb-c70aef51ba1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:18.377103 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676a220-d43b-4fa1-8abb-c70aef51ba1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:18.378075 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676a220-d43b-4fa1-8abb-c70aef51ba1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:18.380059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676a220-d43b-4fa1-8abb-c70aef51ba1b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:18.391795 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1119/2254] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:18.396604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09569a2-4ede-4d53-b171-4567c549a20a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:18.407719 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09569a2-4ede-4d53-b171-4567c549a20a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:18.413032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce2959-ddac-49d1-8c95-0ec95189fd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:18.413698 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce2959-ddac-49d1-8c95-0ec95189fd87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:18.413981 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce2959-ddac-49d1-8c95-0ec95189fd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:18.414472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce2959-ddac-49d1-8c95-0ec95189fd87 admin admin] PATH_INFO: `/v3/projects/62a0d0ab341a47f4aa3ac19826181be5/users/409738291cc742af941dac3276a82faf/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:18.430567 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce2959-ddac-49d1-8c95-0ec95189fd87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a0d0ab341a47f4aa3ac19826181be5, user_id=409738291cc742af941dac3276a82faf, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:18.433278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce2959-ddac-49d1-8c95-0ec95189fd87 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:18.442540 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:18.450175 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1136/2255] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:18 2022] PUT /identity/v3/projects/62a0d0ab341a47f4aa3ac19826181be5/users/409738291cc742af941dac3276a82faf/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:18.460750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c66ce9-8e5b-4b8e-89af-9d88321b929c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:18.461044 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c66ce9-8e5b-4b8e-89af-9d88321b929c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:18.461317 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c66ce9-8e5b-4b8e-89af-9d88321b929c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:18.493422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e0c66ce9-8e5b-4b8e-89af-9d88321b929c None None] MFA Rules not processed for user `409738291cc742af941dac3276a82faf`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:18.494719 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e0c66ce9-8e5b-4b8e-89af-9d88321b929c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:18.578820 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1120/2256] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:18 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:18.584158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce2959-ddac-49d1-8c95-0ec95189fd87 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:18.649764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce2959-ddac-49d1-8c95-0ec95189fd87 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:18.654391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8893f-c5d9-427e-ba15-ee23de57c70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:18.655004 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8893f-c5d9-427e-ba15-ee23de57c70e admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:18.655316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8893f-c5d9-427e-ba15-ee23de57c70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:18.655591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8893f-c5d9-427e-ba15-ee23de57c70e admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:18.656215 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8893f-c5d9-427e-ba15-ee23de57c70e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:18.658958 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8893f-c5d9-427e-ba15-ee23de57c70e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:18.686189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1137/2257] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:18 2022] POST /identity/v3/projects => generated 409 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:18.690250 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c66ce9-8e5b-4b8e-89af-9d88321b929c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:18.701355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c66ce9-8e5b-4b8e-89af-9d88321b929c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:18.730814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:18.768077 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dceed-091f-4df8-a2bd-f515d626ed6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:18.768698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dceed-091f-4df8-a2bd-f515d626ed6d admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:18.768983 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dceed-091f-4df8-a2bd-f515d626ed6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:18.769259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dceed-091f-4df8-a2bd-f515d626ed6d admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:18.769935 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dceed-091f-4df8-a2bd-f515d626ed6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:18.771939 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dceed-091f-4df8-a2bd-f515d626ed6d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:18.802013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1121/2258] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:18 2022] POST /identity/v3/users => generated 311 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:18.806058 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8893f-c5d9-427e-ba15-ee23de57c70e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:18.816909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8893f-c5d9-427e-ba15-ee23de57c70e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:18.821845 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bc75b-ade8-4f56-a2b6-df9d2ef0927a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:18.822839 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bc75b-ade8-4f56-a2b6-df9d2ef0927a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:18.823307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bc75b-ade8-4f56-a2b6-df9d2ef0927a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:18.823736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bc75b-ade8-4f56-a2b6-df9d2ef0927a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:18.824514 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bc75b-ade8-4f56-a2b6-df9d2ef0927a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:18.826278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bc75b-ade8-4f56-a2b6-df9d2ef0927a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:18.838227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1138/2259] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:18.843427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dceed-091f-4df8-a2bd-f515d626ed6d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:18.856810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dceed-091f-4df8-a2bd-f515d626ed6d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:18.862315 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc6b64-4695-45da-b0db-02d32b11452d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:18.862984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc6b64-4695-45da-b0db-02d32b11452d admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:18.863286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc6b64-4695-45da-b0db-02d32b11452d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:18.863554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc6b64-4695-45da-b0db-02d32b11452d admin admin] PATH_INFO: `/v3/projects/6272995ef3fc4e8bb2a7afe6f5f0c828/users/e43d5d16d3614c839a3137da351603b2/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:18.883529 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc6b64-4695-45da-b0db-02d32b11452d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6272995ef3fc4e8bb2a7afe6f5f0c828, user_id=e43d5d16d3614c839a3137da351603b2, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:18.886204 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc6b64-4695-45da-b0db-02d32b11452d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:18.900442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1122/2260] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:18 2022] PUT /identity/v3/projects/6272995ef3fc4e8bb2a7afe6f5f0c828/users/e43d5d16d3614c839a3137da351603b2/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:18.905227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bc75b-ade8-4f56-a2b6-df9d2ef0927a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:18.916495 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bc75b-ade8-4f56-a2b6-df9d2ef0927a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:18.973139 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac631580-0c1e-4a6d-9593-1f4facf02abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:18.973741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac631580-0c1e-4a6d-9593-1f4facf02abf admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:18.974020 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac631580-0c1e-4a6d-9593-1f4facf02abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:18.974292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac631580-0c1e-4a6d-9593-1f4facf02abf admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:18.974929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac631580-0c1e-4a6d-9593-1f4facf02abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:18.976567 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac631580-0c1e-4a6d-9593-1f4facf02abf admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:18.986968 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1139/2261] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:32:18.991564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc6b64-4695-45da-b0db-02d32b11452d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:19.003459 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc6b64-4695-45da-b0db-02d32b11452d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:19.008958 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be8a68-4bcd-458a-9484-3a33969e928a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:19.009712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be8a68-4bcd-458a-9484-3a33969e928a admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:19.010075 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be8a68-4bcd-458a-9484-3a33969e928a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:19.010454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be8a68-4bcd-458a-9484-3a33969e928a admin admin] PATH_INFO: `/v3/projects/6272995ef3fc4e8bb2a7afe6f5f0c828/users/e43d5d16d3614c839a3137da351603b2/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:19.012727 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78be8a68-4bcd-458a-9484-3a33969e928a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6272995ef3fc4e8bb2a7afe6f5f0c828, user_id=e43d5d16d3614c839a3137da351603b2, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:19.016222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78be8a68-4bcd-458a-9484-3a33969e928a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:19.028797 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1123/2262] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:18 2022] PUT /identity/v3/projects/6272995ef3fc4e8bb2a7afe6f5f0c828/users/e43d5d16d3614c839a3137da351603b2/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:19.040672 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5265b1c-177f-4492-99fc-998a6474b5ef None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:19.040988 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5265b1c-177f-4492-99fc-998a6474b5ef None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:19.041266 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5265b1c-177f-4492-99fc-998a6474b5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:19.073031 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f5265b1c-177f-4492-99fc-998a6474b5ef None None] MFA Rules not processed for user `e43d5d16d3614c839a3137da351603b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:19.074293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f5265b1c-177f-4492-99fc-998a6474b5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:19.140135 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:19.153470 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1140/2263] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:19 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:19.164954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be8a68-4bcd-458a-9484-3a33969e928a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:19.166966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-78be8a68-4bcd-458a-9484-3a33969e928a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:19.226932 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be8a68-4bcd-458a-9484-3a33969e928a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:19.232078 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dfb5d-3aaa-41f3-9c21-591cb408943d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:19.232865 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dfb5d-3aaa-41f3-9c21-591cb408943d service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:19.233310 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dfb5d-3aaa-41f3-9c21-591cb408943d service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:19.233687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dfb5d-3aaa-41f3-9c21-591cb408943d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:19.246374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34dfb5d-3aaa-41f3-9c21-591cb408943d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:19.248115 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34dfb5d-3aaa-41f3-9c21-591cb408943d service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:19.262065 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1124/2264] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:19 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:19.426158 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5265b1c-177f-4492-99fc-998a6474b5ef None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:19.436061 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5265b1c-177f-4492-99fc-998a6474b5ef None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:19.440847 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d115f0-d443-403a-a49d-96c6c739d52a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:19.441556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d115f0-d443-403a-a49d-96c6c739d52a service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:19.441936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d115f0-d443-403a-a49d-96c6c739d52a service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:19.442299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d115f0-d443-403a-a49d-96c6c739d52a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:19.501641 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d115f0-d443-403a-a49d-96c6c739d52a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:19.503318 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d115f0-d443-403a-a49d-96c6c739d52a service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:19.512476 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1141/2265] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:19 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:21.079381 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dfb5d-3aaa-41f3-9c21-591cb408943d service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:21.081261 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f34dfb5d-3aaa-41f3-9c21-591cb408943d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:21.159672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dfb5d-3aaa-41f3-9c21-591cb408943d service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:21.163321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73239509-37a2-4e89-b4c6-11e207810ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:21.163954 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73239509-37a2-4e89-b4c6-11e207810ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:21.164285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73239509-37a2-4e89-b4c6-11e207810ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:21.164488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73239509-37a2-4e89-b4c6-11e207810ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:21.166194 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-73239509-37a2-4e89-b4c6-11e207810ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:21.206061 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:21.231366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73239509-37a2-4e89-b4c6-11e207810ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:21.232981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73239509-37a2-4e89-b4c6-11e207810ba9 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:21.244133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1125/2266] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:24.961795 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d115f0-d443-403a-a49d-96c6c739d52a service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:24.963170 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-35d115f0-d443-403a-a49d-96c6c739d52a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:25.034324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73239509-37a2-4e89-b4c6-11e207810ba9 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:25.036341 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-73239509-37a2-4e89-b4c6-11e207810ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:25.108898 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d115f0-d443-403a-a49d-96c6c739d52a service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:25.114612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92614cd-4255-47cf-bd63-1d35191390c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e935ba63cdd4256874b99a2eb0a1a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:25.115388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92614cd-4255-47cf-bd63-1d35191390c3 service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:25.115666 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92614cd-4255-47cf-bd63-1d35191390c3 service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:25.115937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92614cd-4255-47cf-bd63-1d35191390c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:25.117705 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d92614cd-4255-47cf-bd63-1d35191390c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:25.129552 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92614cd-4255-47cf-bd63-1d35191390c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:25.131406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92614cd-4255-47cf-bd63-1d35191390c3 service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:25.133731 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73239509-37a2-4e89-b4c6-11e207810ba9 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:25.143960 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1142/2267] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:25.152574 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac17da-e035-4d85-b0a7-01dd5f576be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:25.153336 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac17da-e035-4d85-b0a7-01dd5f576be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:25.153541 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac17da-e035-4d85-b0a7-01dd5f576be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:25.153783 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac17da-e035-4d85-b0a7-01dd5f576be7 admin admin] PATH_INFO: `/v3/users/e4ebab767c054b289e2279659a1cf2e1` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:25.154959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac17da-e035-4d85-b0a7-01dd5f576be7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4ebab767c054b289e2279659a1cf2e1)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:25.156980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac17da-e035-4d85-b0a7-01dd5f576be7 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:25.211589 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-43ac17da-e035-4d85-b0a7-01dd5f576be7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4ebab767c054b289e2279659a1cf2e1'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:25.218890 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-43ac17da-e035-4d85-b0a7-01dd5f576be7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4ebab767c054b289e2279659a1cf2e1'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:25.219530 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-43ac17da-e035-4d85-b0a7-01dd5f576be7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4ebab767c054b289e2279659a1cf2e1'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:25.232966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-43ac17da-e035-4d85-b0a7-01dd5f576be7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4ebab767c054b289e2279659a1cf2e1'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:25.240529 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-43ac17da-e035-4d85-b0a7-01dd5f576be7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4ebab767c054b289e2279659a1cf2e1'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:25.246536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1126/2268] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:25 2022] DELETE /identity/v3/users/e4ebab767c054b289e2279659a1cf2e1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:25.496553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92614cd-4255-47cf-bd63-1d35191390c3 service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:25.509932 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92614cd-4255-47cf-bd63-1d35191390c3 service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:25.566878 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:25.615100 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99859cae-9d29-4eb5-92e4-b183d267ee1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:25.615629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99859cae-9d29-4eb5-92e4-b183d267ee1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:25.615920 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99859cae-9d29-4eb5-92e4-b183d267ee1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:25.616162 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99859cae-9d29-4eb5-92e4-b183d267ee1f admin admin] PATH_INFO: `/v3/projects/e0d86bdb69fa4c6ca37a301b8c57ec5e` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:25.617324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99859cae-9d29-4eb5-92e4-b183d267ee1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0d86bdb69fa4c6ca37a301b8c57ec5e)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:25.619521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99859cae-9d29-4eb5-92e4-b183d267ee1f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:25.689719 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-99859cae-9d29-4eb5-92e4-b183d267ee1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0d86bdb69fa4c6ca37a301b8c57ec5e', 'request_id': 'req-99859cae-9d29-4eb5-92e4-b183d267ee1f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:25.697623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-99859cae-9d29-4eb5-92e4-b183d267ee1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0d86bdb69fa4c6ca37a301b8c57ec5e', 'request_id': 'req-99859cae-9d29-4eb5-92e4-b183d267ee1f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:25.704851 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-99859cae-9d29-4eb5-92e4-b183d267ee1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0d86bdb69fa4c6ca37a301b8c57ec5e', 'request_id': 'req-99859cae-9d29-4eb5-92e4-b183d267ee1f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:25.722131 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-99859cae-9d29-4eb5-92e4-b183d267ee1f admin admin] The token cache is being invalidate because project e0d86bdb69fa4c6ca37a301b8c57ec5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:25.727887 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-99859cae-9d29-4eb5-92e4-b183d267ee1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:25.730435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-99859cae-9d29-4eb5-92e4-b183d267ee1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:25.732519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1143/2269] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:25 2022] DELETE /identity/v3/projects/e0d86bdb69fa4c6ca37a301b8c57ec5e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:25.742088 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb458e7a-e01a-4401-9567-13b56d813138 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:25.742673 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb458e7a-e01a-4401-9567-13b56d813138 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:25.743183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb458e7a-e01a-4401-9567-13b56d813138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:25.803110 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-eb458e7a-e01a-4401-9567-13b56d813138 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:25.804525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-eb458e7a-e01a-4401-9567-13b56d813138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:25.815388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:25.878868 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1127/2270] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:25.886726 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c8057-8574-4aa3-8c21-68f832286b2e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:25.887015 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c8057-8574-4aa3-8c21-68f832286b2e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:25.887451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c8057-8574-4aa3-8c21-68f832286b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:25.921034 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ce7c8057-8574-4aa3-8c21-68f832286b2e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:25.922644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ce7c8057-8574-4aa3-8c21-68f832286b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:25.932968 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1144/2271] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:25.944451 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb458e7a-e01a-4401-9567-13b56d813138 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:25.954397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb458e7a-e01a-4401-9567-13b56d813138 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:25.959411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4c2db-60fc-4710-8bd7-71d148cc586b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:25.960111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4c2db-60fc-4710-8bd7-71d148cc586b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:25.960464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4c2db-60fc-4710-8bd7-71d148cc586b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:25.960808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4c2db-60fc-4710-8bd7-71d148cc586b admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:25.961533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4c2db-60fc-4710-8bd7-71d148cc586b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:32:25.961909 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4c2db-60fc-4710-8bd7-71d148cc586b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:25.963644 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4c2db-60fc-4710-8bd7-71d148cc586b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:25.977144 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1128/2272] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:32:25.981782 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c8057-8574-4aa3-8c21-68f832286b2e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:25.992207 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c8057-8574-4aa3-8c21-68f832286b2e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:25.998032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d7827-a74f-437e-849b-d9b1afb9be46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:25.999395 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d7827-a74f-437e-849b-d9b1afb9be46 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:25.999548 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d7827-a74f-437e-849b-d9b1afb9be46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:25.999828 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d7827-a74f-437e-849b-d9b1afb9be46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:26.000472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d7827-a74f-437e-849b-d9b1afb9be46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:26.002966 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d7827-a74f-437e-849b-d9b1afb9be46 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:26.034210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1145/2273] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:25 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:32:26.038494 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4c2db-60fc-4710-8bd7-71d148cc586b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:26.052221 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4c2db-60fc-4710-8bd7-71d148cc586b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:26.132591 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729eba1-41d5-42c6-a548-bd16eaba65c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:26.133425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729eba1-41d5-42c6-a548-bd16eaba65c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:26.133807 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729eba1-41d5-42c6-a548-bd16eaba65c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:26.134191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729eba1-41d5-42c6-a548-bd16eaba65c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:26.134944 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729eba1-41d5-42c6-a548-bd16eaba65c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:26.137017 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729eba1-41d5-42c6-a548-bd16eaba65c4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:26.163136 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1129/2274] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:26 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:32:26.167314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d7827-a74f-437e-849b-d9b1afb9be46 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:26.180654 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d7827-a74f-437e-849b-d9b1afb9be46 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:26.185738 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a7f24-5ae7-477c-8c6d-1cbcc84489ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:26.186567 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7f24-5ae7-477c-8c6d-1cbcc84489ec admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:26.186906 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7f24-5ae7-477c-8c6d-1cbcc84489ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:26.187223 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7f24-5ae7-477c-8c6d-1cbcc84489ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:26.187921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a7f24-5ae7-477c-8c6d-1cbcc84489ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:26.189673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a7f24-5ae7-477c-8c6d-1cbcc84489ec admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:26.200327 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1146/2275] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:26.205670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729eba1-41d5-42c6-a548-bd16eaba65c4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:26.216992 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729eba1-41d5-42c6-a548-bd16eaba65c4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:26.224449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45234ea3-ce58-4483-86b2-e97270eaadda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:26.225111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45234ea3-ce58-4483-86b2-e97270eaadda admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:26.225402 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45234ea3-ce58-4483-86b2-e97270eaadda admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:26.225661 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45234ea3-ce58-4483-86b2-e97270eaadda admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:26.226310 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45234ea3-ce58-4483-86b2-e97270eaadda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:26.227966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45234ea3-ce58-4483-86b2-e97270eaadda admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:26.239934 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1130/2276] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:26.245279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a7f24-5ae7-477c-8c6d-1cbcc84489ec admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:26.260527 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a7f24-5ae7-477c-8c6d-1cbcc84489ec admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:26.265406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79e2d4-4209-4de0-bdb9-bca60e1ad815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:26.266075 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79e2d4-4209-4de0-bdb9-bca60e1ad815 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:26.266467 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79e2d4-4209-4de0-bdb9-bca60e1ad815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:26.267309 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79e2d4-4209-4de0-bdb9-bca60e1ad815 admin admin] PATH_INFO: `/v3/projects/e5f99554d6ee4433b3df3e92eabae5c2/users/b0f80775472b4e20b22d8142d9c8f6bf/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:26.283554 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79e2d4-4209-4de0-bdb9-bca60e1ad815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5f99554d6ee4433b3df3e92eabae5c2, user_id=b0f80775472b4e20b22d8142d9c8f6bf, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:26.286245 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79e2d4-4209-4de0-bdb9-bca60e1ad815 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:26.303716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1147/2277] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:26 2022] PUT /identity/v3/projects/e5f99554d6ee4433b3df3e92eabae5c2/users/b0f80775472b4e20b22d8142d9c8f6bf/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:26.316082 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da637a66-eb51-4e26-8150-70204fb8f95a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:26.316429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da637a66-eb51-4e26-8150-70204fb8f95a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:26.316764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da637a66-eb51-4e26-8150-70204fb8f95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:26.346674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-da637a66-eb51-4e26-8150-70204fb8f95a None None] MFA Rules not processed for user `b0f80775472b4e20b22d8142d9c8f6bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:26.347896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-da637a66-eb51-4e26-8150-70204fb8f95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:26.416880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1131/2278] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:26.421842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79e2d4-4209-4de0-bdb9-bca60e1ad815 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:26.487957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79e2d4-4209-4de0-bdb9-bca60e1ad815 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:26.492759 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ec49b-3c8f-46b2-8a93-a4a60867cd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:26.493371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ec49b-3c8f-46b2-8a93-a4a60867cd20 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:26.493652 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ec49b-3c8f-46b2-8a93-a4a60867cd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:26.493925 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ec49b-3c8f-46b2-8a93-a4a60867cd20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:26.494565 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ec49b-3c8f-46b2-8a93-a4a60867cd20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:26.496563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ec49b-3c8f-46b2-8a93-a4a60867cd20 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:26.501775 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:26.526062 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1148/2279] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:26 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:26.531327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da637a66-eb51-4e26-8150-70204fb8f95a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:26.545452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da637a66-eb51-4e26-8150-70204fb8f95a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:26.551417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:26.630981 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9b325-98bd-4562-b53a-f3c5bb3ae824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:26.631727 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9b325-98bd-4562-b53a-f3c5bb3ae824 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:26.632081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9b325-98bd-4562-b53a-f3c5bb3ae824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:26.632454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9b325-98bd-4562-b53a-f3c5bb3ae824 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:26.633168 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9b325-98bd-4562-b53a-f3c5bb3ae824 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:26.635444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9b325-98bd-4562-b53a-f3c5bb3ae824 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:26.661853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1132/2280] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:26 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:26.665745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ec49b-3c8f-46b2-8a93-a4a60867cd20 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:26.676672 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ec49b-3c8f-46b2-8a93-a4a60867cd20 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:26.681589 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3d1c7-7fb8-46ea-93fb-6a950705dc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:26.682094 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3d1c7-7fb8-46ea-93fb-6a950705dc48 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:26.682372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3d1c7-7fb8-46ea-93fb-6a950705dc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:26.682650 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3d1c7-7fb8-46ea-93fb-6a950705dc48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:26.683330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3d1c7-7fb8-46ea-93fb-6a950705dc48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:26.684926 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3d1c7-7fb8-46ea-93fb-6a950705dc48 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:26.695604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1149/2281] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:32:26.699637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9b325-98bd-4562-b53a-f3c5bb3ae824 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:26.710450 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9b325-98bd-4562-b53a-f3c5bb3ae824 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:26.715853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebd5f2-ab09-4d18-b5a0-5717320b8073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:26.717699 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebd5f2-ab09-4d18-b5a0-5717320b8073 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:26.718586 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebd5f2-ab09-4d18-b5a0-5717320b8073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:26.718900 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebd5f2-ab09-4d18-b5a0-5717320b8073 admin admin] PATH_INFO: `/v3/projects/85e812d1942e44f7a2e9a25839c09153/users/0b97696221e74f10b03e62672baf85f4/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:26.739620 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cebd5f2-ab09-4d18-b5a0-5717320b8073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85e812d1942e44f7a2e9a25839c09153, user_id=0b97696221e74f10b03e62672baf85f4, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:26.742301 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cebd5f2-ab09-4d18-b5a0-5717320b8073 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:26.756142 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1133/2282] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:26 2022] PUT /identity/v3/projects/85e812d1942e44f7a2e9a25839c09153/users/0b97696221e74f10b03e62672baf85f4/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:26.767234 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3d1c7-7fb8-46ea-93fb-6a950705dc48 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:26.784067 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3d1c7-7fb8-46ea-93fb-6a950705dc48 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:26.850639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4fcdf-0289-44a7-b6d0-60a98601415c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:26.851299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4fcdf-0289-44a7-b6d0-60a98601415c admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:26.851583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4fcdf-0289-44a7-b6d0-60a98601415c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:26.851858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4fcdf-0289-44a7-b6d0-60a98601415c admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:26.852503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4fcdf-0289-44a7-b6d0-60a98601415c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:26.854093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4fcdf-0289-44a7-b6d0-60a98601415c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:26.866765 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1150/2283] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:32:26.870760 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebd5f2-ab09-4d18-b5a0-5717320b8073 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:26.883098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebd5f2-ab09-4d18-b5a0-5717320b8073 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:26.889678 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4296684-f220-4888-8aca-bbdf27ac07e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:26.890460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4296684-f220-4888-8aca-bbdf27ac07e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:26.890844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4296684-f220-4888-8aca-bbdf27ac07e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:26.891221 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4296684-f220-4888-8aca-bbdf27ac07e4 admin admin] PATH_INFO: `/v3/projects/85e812d1942e44f7a2e9a25839c09153/users/0b97696221e74f10b03e62672baf85f4/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:26.893466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4296684-f220-4888-8aca-bbdf27ac07e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85e812d1942e44f7a2e9a25839c09153, user_id=0b97696221e74f10b03e62672baf85f4, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:26.896378 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4296684-f220-4888-8aca-bbdf27ac07e4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:26.907480 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1134/2284] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:26 2022] PUT /identity/v3/projects/85e812d1942e44f7a2e9a25839c09153/users/0b97696221e74f10b03e62672baf85f4/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:26.918091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b9576-3f4e-4611-a5bc-2655d405529e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:26.918409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b9576-3f4e-4611-a5bc-2655d405529e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:26.918685 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b9576-3f4e-4611-a5bc-2655d405529e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:26.950628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d61b9576-3f4e-4611-a5bc-2655d405529e None None] MFA Rules not processed for user `0b97696221e74f10b03e62672baf85f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:26.951851 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d61b9576-3f4e-4611-a5bc-2655d405529e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:27.035192 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1151/2285] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:26 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:27.047460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4296684-f220-4888-8aca-bbdf27ac07e4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:27.049684 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a4296684-f220-4888-8aca-bbdf27ac07e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:27.124001 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4296684-f220-4888-8aca-bbdf27ac07e4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:27.128568 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35827a-64de-4f99-a56f-ac0a2d9de944 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:27.129299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35827a-64de-4f99-a56f-ac0a2d9de944 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:27.129584 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35827a-64de-4f99-a56f-ac0a2d9de944 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:27.129860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35827a-64de-4f99-a56f-ac0a2d9de944 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:27.140059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35827a-64de-4f99-a56f-ac0a2d9de944 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:27.141119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35827a-64de-4f99-a56f-ac0a2d9de944 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:27.150097 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b9576-3f4e-4611-a5bc-2655d405529e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:27.152020 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1135/2286] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:27.152322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d61b9576-3f4e-4611-a5bc-2655d405529e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:27.250914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:27.258489 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b9576-3f4e-4611-a5bc-2655d405529e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:27.264398 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031a4f6-7cf9-40f5-a4ae-39f1fa928754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:27.265056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031a4f6-7cf9-40f5-a4ae-39f1fa928754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:27.265336 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031a4f6-7cf9-40f5-a4ae-39f1fa928754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:27.265713 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031a4f6-7cf9-40f5-a4ae-39f1fa928754 admin admin] PATH_INFO: `/v3/users/409738291cc742af941dac3276a82faf` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:27.267078 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031a4f6-7cf9-40f5-a4ae-39f1fa928754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=409738291cc742af941dac3276a82faf)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:27.269108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031a4f6-7cf9-40f5-a4ae-39f1fa928754 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:27.282164 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35827a-64de-4f99-a56f-ac0a2d9de944 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:27.294854 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35827a-64de-4f99-a56f-ac0a2d9de944 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:27.299448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5855fb-e593-4e5d-8abd-9a3dafe60d18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:27.300085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5855fb-e593-4e5d-8abd-9a3dafe60d18 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:27.300509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5855fb-e593-4e5d-8abd-9a3dafe60d18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:27.300786 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5855fb-e593-4e5d-8abd-9a3dafe60d18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:27.351383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2031a4f6-7cf9-40f5-a4ae-39f1fa928754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '409738291cc742af941dac3276a82faf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:27.362608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2031a4f6-7cf9-40f5-a4ae-39f1fa928754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '409738291cc742af941dac3276a82faf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:27.379934 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2031a4f6-7cf9-40f5-a4ae-39f1fa928754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '409738291cc742af941dac3276a82faf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:27.391757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2031a4f6-7cf9-40f5-a4ae-39f1fa928754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '409738291cc742af941dac3276a82faf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:27.392944 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2031a4f6-7cf9-40f5-a4ae-39f1fa928754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '409738291cc742af941dac3276a82faf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:27.402920 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1152/2287] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:27 2022] DELETE /identity/v3/users/409738291cc742af941dac3276a82faf => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:27.407895 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bc86a-440b-4067-98b4-8c6a12ae1fac None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:27.408361 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bc86a-440b-4067-98b4-8c6a12ae1fac None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:27.408924 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bc86a-440b-4067-98b4-8c6a12ae1fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:27.423715 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5855fb-e593-4e5d-8abd-9a3dafe60d18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:27.425492 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5855fb-e593-4e5d-8abd-9a3dafe60d18 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:27.441440 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-252bc86a-440b-4067-98b4-8c6a12ae1fac None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:27.443455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-252bc86a-440b-4067-98b4-8c6a12ae1fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:27.444715 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1136/2288] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:27.515290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1153/2289] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:32:27.526546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5855fb-e593-4e5d-8abd-9a3dafe60d18 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:27.528464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8c5855fb-e593-4e5d-8abd-9a3dafe60d18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:27.535567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:27.607253 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5855fb-e593-4e5d-8abd-9a3dafe60d18 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:27.677103 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be39dd1-ade5-4877-9e46-562a448a0b82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:27.677891 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be39dd1-ade5-4877-9e46-562a448a0b82 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:27.678105 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be39dd1-ade5-4877-9e46-562a448a0b82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:27.678343 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be39dd1-ade5-4877-9e46-562a448a0b82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:27.690497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be39dd1-ade5-4877-9e46-562a448a0b82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:27.692000 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be39dd1-ade5-4877-9e46-562a448a0b82 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:27.702542 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1137/2290] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:28.012145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bc86a-440b-4067-98b4-8c6a12ae1fac None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:28.014017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-252bc86a-440b-4067-98b4-8c6a12ae1fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:28.028144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bc86a-440b-4067-98b4-8c6a12ae1fac None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:28.033923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:28.034555 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:28.034827 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:28.035126 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] PATH_INFO: `/v3/projects/62a0d0ab341a47f4aa3ac19826181be5` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:28.037306 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a0d0ab341a47f4aa3ac19826181be5)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:28.039380 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:28.085340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a0d0ab341a47f4aa3ac19826181be5', 'request_id': 'req-75f87026-3460-4716-b2a7-eed23edf9ff4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.090777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a0d0ab341a47f4aa3ac19826181be5', 'request_id': 'req-75f87026-3460-4716-b2a7-eed23edf9ff4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.097676 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a0d0ab341a47f4aa3ac19826181be5', 'request_id': 'req-75f87026-3460-4716-b2a7-eed23edf9ff4'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.107547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:28.113024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] The token cache is being invalidate because project 62a0d0ab341a47f4aa3ac19826181be5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:28.113479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.114449 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.116188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1154/2291] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:28 2022] DELETE /identity/v3/projects/62a0d0ab341a47f4aa3ac19826181be5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:28.120346 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be39dd1-ade5-4877-9e46-562a448a0b82 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:28.130562 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be39dd1-ade5-4877-9e46-562a448a0b82 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:28.187688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922b3da-3a72-4916-a31a-5202bbaa14a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:28.188322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922b3da-3a72-4916-a31a-5202bbaa14a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:28.188698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922b3da-3a72-4916-a31a-5202bbaa14a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:28.189145 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922b3da-3a72-4916-a31a-5202bbaa14a1 admin admin] PATH_INFO: `/v3/users/e43d5d16d3614c839a3137da351603b2` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:28.190289 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922b3da-3a72-4916-a31a-5202bbaa14a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e43d5d16d3614c839a3137da351603b2)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:28.192232 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922b3da-3a72-4916-a31a-5202bbaa14a1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:28.239768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b922b3da-3a72-4916-a31a-5202bbaa14a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e43d5d16d3614c839a3137da351603b2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.247095 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b922b3da-3a72-4916-a31a-5202bbaa14a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e43d5d16d3614c839a3137da351603b2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.247699 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b922b3da-3a72-4916-a31a-5202bbaa14a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e43d5d16d3614c839a3137da351603b2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.254724 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:28.261584 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b922b3da-3a72-4916-a31a-5202bbaa14a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e43d5d16d3614c839a3137da351603b2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.269038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b922b3da-3a72-4916-a31a-5202bbaa14a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e43d5d16d3614c839a3137da351603b2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.274973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1138/2292] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:28 2022] DELETE /identity/v3/users/e43d5d16d3614c839a3137da351603b2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:28.542436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:28.544185 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:28.569455 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f87026-3460-4716-b2a7-eed23edf9ff4 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:28.630652 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:28.631312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:28.631588 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:28.631853 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae admin admin] PATH_INFO: `/v3/projects/6272995ef3fc4e8bb2a7afe6f5f0c828` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:28.632978 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=6272995ef3fc4e8bb2a7afe6f5f0c828)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:28.634947 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:28.681009 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6272995ef3fc4e8bb2a7afe6f5f0c828', 'request_id': 'req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.686444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6272995ef3fc4e8bb2a7afe6f5f0c828', 'request_id': 'req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.693060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6272995ef3fc4e8bb2a7afe6f5f0c828', 'request_id': 'req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.706160 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae admin admin] The token cache is being invalidate because project 6272995ef3fc4e8bb2a7afe6f5f0c828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:28.706447 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.707092 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c0eb1adf-e139-40a4-b349-89c67c6bd4ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:28.708592 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1155/2293] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:28 2022] DELETE /identity/v3/projects/6272995ef3fc4e8bb2a7afe6f5f0c828 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:28.713654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb50f1e-32bf-4fe5-8c1c-e9ea49693c66 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:28.713943 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb50f1e-32bf-4fe5-8c1c-e9ea49693c66 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:28.714218 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb50f1e-32bf-4fe5-8c1c-e9ea49693c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:28.735334 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-fcb50f1e-32bf-4fe5-8c1c-e9ea49693c66 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:28.736706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-fcb50f1e-32bf-4fe5-8c1c-e9ea49693c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:28.802264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1139/2294] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:28.810171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28451c-5578-4e9a-bcc7-f300fd62b5ca None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:28.810456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28451c-5578-4e9a-bcc7-f300fd62b5ca None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:28.810733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28451c-5578-4e9a-bcc7-f300fd62b5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:28.832112 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-7a28451c-5578-4e9a-bcc7-f300fd62b5ca None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:28.833319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7a28451c-5578-4e9a-bcc7-f300fd62b5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:28.841227 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1156/2295] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:28.845820 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb50f1e-32bf-4fe5-8c1c-e9ea49693c66 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:28.855533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb50f1e-32bf-4fe5-8c1c-e9ea49693c66 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:28.860538 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33e6d2-3d60-49a6-8173-4c14e51c83c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:28.861143 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33e6d2-3d60-49a6-8173-4c14e51c83c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:28.861427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33e6d2-3d60-49a6-8173-4c14e51c83c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:28.861697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33e6d2-3d60-49a6-8173-4c14e51c83c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:28.862362 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33e6d2-3d60-49a6-8173-4c14e51c83c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:32:28.862661 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33e6d2-3d60-49a6-8173-4c14e51c83c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:28.864293 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33e6d2-3d60-49a6-8173-4c14e51c83c8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:28.880245 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1140/2296] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:32:28.881776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28451c-5578-4e9a-bcc7-f300fd62b5ca None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:28.890979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28451c-5578-4e9a-bcc7-f300fd62b5ca None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:28.899446 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87e7b0-7b95-4a99-801d-96a316721832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:28.900092 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87e7b0-7b95-4a99-801d-96a316721832 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:28.900377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87e7b0-7b95-4a99-801d-96a316721832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:28.900654 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87e7b0-7b95-4a99-801d-96a316721832 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:28.901291 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87e7b0-7b95-4a99-801d-96a316721832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:28.903231 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87e7b0-7b95-4a99-801d-96a316721832 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:28.931214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1157/2297] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:28 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:28.935538 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33e6d2-3d60-49a6-8173-4c14e51c83c8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:28.946512 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33e6d2-3d60-49a6-8173-4c14e51c83c8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:29.007589 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67367082-7909-4655-b1d0-4744bae81907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:29.008210 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67367082-7909-4655-b1d0-4744bae81907 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:29.008487 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67367082-7909-4655-b1d0-4744bae81907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:29.008756 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67367082-7909-4655-b1d0-4744bae81907 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:29.009378 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67367082-7909-4655-b1d0-4744bae81907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:29.011312 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67367082-7909-4655-b1d0-4744bae81907 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:29.033562 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1141/2298] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:28 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:32:29.037658 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87e7b0-7b95-4a99-801d-96a316721832 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:29.048282 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87e7b0-7b95-4a99-801d-96a316721832 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:29.053466 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc248bf-7d5d-4864-b92f-cfb59d1305d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:29.054148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc248bf-7d5d-4864-b92f-cfb59d1305d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:29.054557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc248bf-7d5d-4864-b92f-cfb59d1305d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:29.054923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc248bf-7d5d-4864-b92f-cfb59d1305d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:29.055697 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc248bf-7d5d-4864-b92f-cfb59d1305d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:29.057561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc248bf-7d5d-4864-b92f-cfb59d1305d9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:29.074922 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1158/2299] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:29.076445 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67367082-7909-4655-b1d0-4744bae81907 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:29.088851 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67367082-7909-4655-b1d0-4744bae81907 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:29.094560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3720975-7b14-4be2-a828-43bc0cec03d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:29.095254 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3720975-7b14-4be2-a828-43bc0cec03d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:29.095539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3720975-7b14-4be2-a828-43bc0cec03d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:29.095814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3720975-7b14-4be2-a828-43bc0cec03d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:29.096460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3720975-7b14-4be2-a828-43bc0cec03d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:29.098053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3720975-7b14-4be2-a828-43bc0cec03d9 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:29.110338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1142/2300] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:29.114393 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc248bf-7d5d-4864-b92f-cfb59d1305d9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:29.124586 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc248bf-7d5d-4864-b92f-cfb59d1305d9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:29.129743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5c4f2-a700-41f4-97cb-f8c2b3d048c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:29.130426 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5c4f2-a700-41f4-97cb-f8c2b3d048c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:29.130757 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5c4f2-a700-41f4-97cb-f8c2b3d048c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:29.130969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5c4f2-a700-41f4-97cb-f8c2b3d048c8 admin admin] PATH_INFO: `/v3/projects/39257b45e242410ba055d41edd921288/users/f9c17d0e2f274942abd77a90bb7d55d6/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:29.145948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5c4f2-a700-41f4-97cb-f8c2b3d048c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39257b45e242410ba055d41edd921288, user_id=f9c17d0e2f274942abd77a90bb7d55d6, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:29.148636 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5c4f2-a700-41f4-97cb-f8c2b3d048c8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:29.161159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1159/2301] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:29 2022] PUT /identity/v3/projects/39257b45e242410ba055d41edd921288/users/f9c17d0e2f274942abd77a90bb7d55d6/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:29.171500 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bf9dc-a486-4cf6-9a41-bd3ff071425a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:29.171963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bf9dc-a486-4cf6-9a41-bd3ff071425a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:29.172060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bf9dc-a486-4cf6-9a41-bd3ff071425a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:29.200233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-922bf9dc-a486-4cf6-9a41-bd3ff071425a None None] MFA Rules not processed for user `f9c17d0e2f274942abd77a90bb7d55d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:29.201438 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-922bf9dc-a486-4cf6-9a41-bd3ff071425a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:29.236066 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:29.269646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1143/2302] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:32:29.274308 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5c4f2-a700-41f4-97cb-f8c2b3d048c8 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:29.302397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:29.340059 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5c4f2-a700-41f4-97cb-f8c2b3d048c8 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:29.344653 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef07ea3-fd26-4fd9-8c9f-5a156b1d826f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:29.345466 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef07ea3-fd26-4fd9-8c9f-5a156b1d826f admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:29.345746 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef07ea3-fd26-4fd9-8c9f-5a156b1d826f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:29.346018 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef07ea3-fd26-4fd9-8c9f-5a156b1d826f admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:29.346633 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef07ea3-fd26-4fd9-8c9f-5a156b1d826f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:29.348555 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef07ea3-fd26-4fd9-8c9f-5a156b1d826f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:29.375370 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1160/2303] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:29 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:29.379994 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bf9dc-a486-4cf6-9a41-bd3ff071425a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:29.391345 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bf9dc-a486-4cf6-9a41-bd3ff071425a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:29.446308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c082a-f364-4174-95df-5cc737889cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:29.446916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c082a-f364-4174-95df-5cc737889cfd admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:29.447226 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c082a-f364-4174-95df-5cc737889cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:29.447499 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c082a-f364-4174-95df-5cc737889cfd admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:29.448117 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c082a-f364-4174-95df-5cc737889cfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:29.450009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c082a-f364-4174-95df-5cc737889cfd admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:29.471146 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1144/2304] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:29 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:29.475117 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef07ea3-fd26-4fd9-8c9f-5a156b1d826f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:29.486341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef07ea3-fd26-4fd9-8c9f-5a156b1d826f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:29.491032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12baf701-a8a0-4e8b-abd8-e1ef5b0a376f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:29.491657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12baf701-a8a0-4e8b-abd8-e1ef5b0a376f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:29.491940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12baf701-a8a0-4e8b-abd8-e1ef5b0a376f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:29.492213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12baf701-a8a0-4e8b-abd8-e1ef5b0a376f admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:29.492850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12baf701-a8a0-4e8b-abd8-e1ef5b0a376f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:29.494449 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12baf701-a8a0-4e8b-abd8-e1ef5b0a376f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:29.507268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1161/2305] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:29.511149 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c082a-f364-4174-95df-5cc737889cfd admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:29.521605 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c082a-f364-4174-95df-5cc737889cfd admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:29.526129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e3afd-26c0-4018-b882-9f2a28c77283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:29.526720 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e3afd-26c0-4018-b882-9f2a28c77283 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:29.526998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e3afd-26c0-4018-b882-9f2a28c77283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:29.527299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e3afd-26c0-4018-b882-9f2a28c77283 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:29.527929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e3afd-26c0-4018-b882-9f2a28c77283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:29.529823 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e3afd-26c0-4018-b882-9f2a28c77283 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:29.540557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1145/2306] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:29.544815 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12baf701-a8a0-4e8b-abd8-e1ef5b0a376f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:29.555343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12baf701-a8a0-4e8b-abd8-e1ef5b0a376f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:29.560278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030f4e1-cb57-463a-b5ba-7c126f0c1641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:29.560923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f4e1-cb57-463a-b5ba-7c126f0c1641 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:29.561201 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f4e1-cb57-463a-b5ba-7c126f0c1641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:29.561473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f4e1-cb57-463a-b5ba-7c126f0c1641 admin admin] PATH_INFO: `/v3/projects/21aadf53c1424237b34422bfcd93f50f/users/d66b1db5a691431ea7ee2ee4e1ad1bea/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:29.576501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030f4e1-cb57-463a-b5ba-7c126f0c1641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21aadf53c1424237b34422bfcd93f50f, user_id=d66b1db5a691431ea7ee2ee4e1ad1bea, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:29.579174 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030f4e1-cb57-463a-b5ba-7c126f0c1641 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:29.591722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1162/2307] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:29 2022] PUT /identity/v3/projects/21aadf53c1424237b34422bfcd93f50f/users/d66b1db5a691431ea7ee2ee4e1ad1bea/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:29.659705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606917a-260b-45af-a98d-6a1d1dc1316b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:29.659990 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606917a-260b-45af-a98d-6a1d1dc1316b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:29.660264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606917a-260b-45af-a98d-6a1d1dc1316b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:29.689672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-8606917a-260b-45af-a98d-6a1d1dc1316b None None] MFA Rules not processed for user `d66b1db5a691431ea7ee2ee4e1ad1bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:29.691171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8606917a-260b-45af-a98d-6a1d1dc1316b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:29.765506 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1146/2308] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:29.770396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030f4e1-cb57-463a-b5ba-7c126f0c1641 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:29.841888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030f4e1-cb57-463a-b5ba-7c126f0c1641 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:29.846954 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302c53f-3f57-4f93-ab26-1400c6bf130f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:29.847638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c53f-3f57-4f93-ab26-1400c6bf130f admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:29.847918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c53f-3f57-4f93-ab26-1400c6bf130f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:29.848189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c53f-3f57-4f93-ab26-1400c6bf130f admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:29.848816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302c53f-3f57-4f93-ab26-1400c6bf130f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:29.850741 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302c53f-3f57-4f93-ab26-1400c6bf130f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:29.877332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1163/2309] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:29 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:29.881371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606917a-260b-45af-a98d-6a1d1dc1316b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:29.892937 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606917a-260b-45af-a98d-6a1d1dc1316b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:29.935507 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:29.951404 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893b275-a05b-4cdf-adf3-70a523789f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:29.952025 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893b275-a05b-4cdf-adf3-70a523789f2f admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:29.952375 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893b275-a05b-4cdf-adf3-70a523789f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:29.952683 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893b275-a05b-4cdf-adf3-70a523789f2f admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:29.953320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893b275-a05b-4cdf-adf3-70a523789f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:29.955849 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893b275-a05b-4cdf-adf3-70a523789f2f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:29.976396 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1147/2310] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:29 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:29.980481 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302c53f-3f57-4f93-ab26-1400c6bf130f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:29.991165 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302c53f-3f57-4f93-ab26-1400c6bf130f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:29.995858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693ca57-6b70-4549-84a1-d8fcc6b38457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:29.996461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693ca57-6b70-4549-84a1-d8fcc6b38457 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:29.996752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693ca57-6b70-4549-84a1-d8fcc6b38457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:29.997025 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693ca57-6b70-4549-84a1-d8fcc6b38457 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:29.997688 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693ca57-6b70-4549-84a1-d8fcc6b38457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:29.999333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693ca57-6b70-4549-84a1-d8fcc6b38457 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:30.010925 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1164/2311] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:30.015165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893b275-a05b-4cdf-adf3-70a523789f2f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:30.025656 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893b275-a05b-4cdf-adf3-70a523789f2f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:30.030251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ff4a6-d66b-4b22-8587-4b7a4a6b559e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:30.030900 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ff4a6-d66b-4b22-8587-4b7a4a6b559e admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:30.031214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ff4a6-d66b-4b22-8587-4b7a4a6b559e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:30.031489 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ff4a6-d66b-4b22-8587-4b7a4a6b559e admin admin] PATH_INFO: `/v3/projects/45d6cfc1e16b4faba4ffe8509f45cb66/users/3f35a912286744d591df162545b9f578/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:30.047260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ff4a6-d66b-4b22-8587-4b7a4a6b559e admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d6cfc1e16b4faba4ffe8509f45cb66, user_id=3f35a912286744d591df162545b9f578, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:30.049988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ff4a6-d66b-4b22-8587-4b7a4a6b559e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:30.062409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1148/2312] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:30 2022] PUT /identity/v3/projects/45d6cfc1e16b4faba4ffe8509f45cb66/users/3f35a912286744d591df162545b9f578/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:30.067199 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693ca57-6b70-4549-84a1-d8fcc6b38457 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:30.077073 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693ca57-6b70-4549-84a1-d8fcc6b38457 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:30.121732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:30.137722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc3eb4-7691-40f5-b420-d477d8601eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:30.138438 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc3eb4-7691-40f5-b420-d477d8601eda admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:30.138784 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc3eb4-7691-40f5-b420-d477d8601eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:30.139139 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc3eb4-7691-40f5-b420-d477d8601eda admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:30.139851 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc3eb4-7691-40f5-b420-d477d8601eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:30.141573 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc3eb4-7691-40f5-b420-d477d8601eda admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:30.148337 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ff4a6-d66b-4b22-8587-4b7a4a6b559e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:30.150061 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bf7ff4a6-d66b-4b22-8587-4b7a4a6b559e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:30.152610 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1165/2313] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:30.156361 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc3eb4-7691-40f5-b420-d477d8601eda admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:30.163111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ff4a6-d66b-4b22-8587-4b7a4a6b559e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:30.166622 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc3eb4-7691-40f5-b420-d477d8601eda admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:30.167720 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c362adf-b02b-4534-9d50-1842c1a6f020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:30.168341 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c362adf-b02b-4534-9d50-1842c1a6f020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:30.168615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c362adf-b02b-4534-9d50-1842c1a6f020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:30.168885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c362adf-b02b-4534-9d50-1842c1a6f020 admin admin] PATH_INFO: `/v3/users/b0f80775472b4e20b22d8142d9c8f6bf` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:30.170000 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c362adf-b02b-4534-9d50-1842c1a6f020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0f80775472b4e20b22d8142d9c8f6bf)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:30.171400 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8a9a8-ac85-48f8-84ea-9c49b79b30a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:30.171935 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c362adf-b02b-4534-9d50-1842c1a6f020 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:30.172387 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8a9a8-ac85-48f8-84ea-9c49b79b30a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:30.172387 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8a9a8-ac85-48f8-84ea-9c49b79b30a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:30.172588 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8a9a8-ac85-48f8-84ea-9c49b79b30a1 admin admin] PATH_INFO: `/v3/projects/45d6cfc1e16b4faba4ffe8509f45cb66/users/3f35a912286744d591df162545b9f578/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:30.174965 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8a9a8-ac85-48f8-84ea-9c49b79b30a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d6cfc1e16b4faba4ffe8509f45cb66, user_id=3f35a912286744d591df162545b9f578, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:30.177804 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8a9a8-ac85-48f8-84ea-9c49b79b30a1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:30.188177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1166/2314] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:30 2022] PUT /identity/v3/projects/45d6cfc1e16b4faba4ffe8509f45cb66/users/3f35a912286744d591df162545b9f578/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:30.199433 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8da92-b60b-4421-9f24-f2e02281b3bb None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:30.200204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8da92-b60b-4421-9f24-f2e02281b3bb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:30.200492 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8da92-b60b-4421-9f24-f2e02281b3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:30.224899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3c362adf-b02b-4534-9d50-1842c1a6f020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0f80775472b4e20b22d8142d9c8f6bf'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:30.231149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-15c8da92-b60b-4421-9f24-f2e02281b3bb None None] MFA Rules not processed for user `3f35a912286744d591df162545b9f578`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:30.232754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-15c8da92-b60b-4421-9f24-f2e02281b3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:30.234753 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3c362adf-b02b-4534-9d50-1842c1a6f020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0f80775472b4e20b22d8142d9c8f6bf'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:30.236102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3c362adf-b02b-4534-9d50-1842c1a6f020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0f80775472b4e20b22d8142d9c8f6bf'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:30.254640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3c362adf-b02b-4534-9d50-1842c1a6f020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0f80775472b4e20b22d8142d9c8f6bf'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:30.262568 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3c362adf-b02b-4534-9d50-1842c1a6f020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0f80775472b4e20b22d8142d9c8f6bf'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:30.268824 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1149/2315] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:30 2022] DELETE /identity/v3/users/b0f80775472b4e20b22d8142d9c8f6bf => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:30.273022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28561d-76da-4242-aaf1-4ca7e267a83d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:30.273304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28561d-76da-4242-aaf1-4ca7e267a83d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:30.273572 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28561d-76da-4242-aaf1-4ca7e267a83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:30.291834 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ed28561d-76da-4242-aaf1-4ca7e267a83d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:30.293043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ed28561d-76da-4242-aaf1-4ca7e267a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:30.306211 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1167/2316] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:32:30.318216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8da92-b60b-4421-9f24-f2e02281b3bb None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:30.319551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-15c8da92-b60b-4421-9f24-f2e02281b3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:30.352763 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1150/2317] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:32:30.358363 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28561d-76da-4242-aaf1-4ca7e267a83d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:30.359974 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ed28561d-76da-4242-aaf1-4ca7e267a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:30.381816 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8da92-b60b-4421-9f24-f2e02281b3bb None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:30.387688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:30.421657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28561d-76da-4242-aaf1-4ca7e267a83d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:30.426072 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa47b2-9739-4cce-bc45-9f12042c29ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:30.426685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa47b2-9739-4cce-bc45-9f12042c29ad service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:30.426955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa47b2-9739-4cce-bc45-9f12042c29ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:30.427251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa47b2-9739-4cce-bc45-9f12042c29ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:30.429029 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-71aa47b2-9739-4cce-bc45-9f12042c29ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:30.432091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0be6d-5186-46c0-b5e3-5e040b52b480 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:30.432706 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0be6d-5186-46c0-b5e3-5e040b52b480 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:30.432991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0be6d-5186-46c0-b5e3-5e040b52b480 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:30.433263 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0be6d-5186-46c0-b5e3-5e040b52b480 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:30.435052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-72c0be6d-5186-46c0-b5e3-5e040b52b480 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:30.464160 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:30.490114 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa47b2-9739-4cce-bc45-9f12042c29ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:30.491637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa47b2-9739-4cce-bc45-9f12042c29ad service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:30.498887 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0be6d-5186-46c0-b5e3-5e040b52b480 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:30.500349 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0be6d-5186-46c0-b5e3-5e040b52b480 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:30.500834 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1151/2318] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:32:30.501640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa47b2-9739-4cce-bc45-9f12042c29ad service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:30.502990 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-71aa47b2-9739-4cce-bc45-9f12042c29ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:30.511930 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1168/2319] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:32:30.515390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa47b2-9739-4cce-bc45-9f12042c29ad service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:30.519888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652e410-e0ba-41bb-92e6-9138f7a67409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:30.520581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652e410-e0ba-41bb-92e6-9138f7a67409 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:30.520757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652e410-e0ba-41bb-92e6-9138f7a67409 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:30.521063 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652e410-e0ba-41bb-92e6-9138f7a67409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:30.531266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652e410-e0ba-41bb-92e6-9138f7a67409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:30.533392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652e410-e0ba-41bb-92e6-9138f7a67409 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:30.544272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1152/2320] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:30.870461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0be6d-5186-46c0-b5e3-5e040b52b480 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:30.871931 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-72c0be6d-5186-46c0-b5e3-5e040b52b480 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:30.895035 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0be6d-5186-46c0-b5e3-5e040b52b480 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:30.902226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:30.902961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:30.903339 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:30.903745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] PATH_INFO: `/v3/projects/e5f99554d6ee4433b3df3e92eabae5c2` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:30.905034 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5f99554d6ee4433b3df3e92eabae5c2)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:30.909996 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:30.969156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652e410-e0ba-41bb-92e6-9138f7a67409 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:30.980421 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652e410-e0ba-41bb-92e6-9138f7a67409 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:30.984999 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf09e4-6a4f-4f4b-9744-cb9d119f59c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:30.985654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf09e4-6a4f-4f4b-9744-cb9d119f59c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:30.985957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf09e4-6a4f-4f4b-9744-cb9d119f59c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:30.986227 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf09e4-6a4f-4f4b-9744-cb9d119f59c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:30.988009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d9cf09e4-6a4f-4f4b-9744-cb9d119f59c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:31.000536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf09e4-6a4f-4f4b-9744-cb9d119f59c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:31.002023 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf09e4-6a4f-4f4b-9744-cb9d119f59c6 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:31.010445 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5f99554d6ee4433b3df3e92eabae5c2', 'request_id': 'req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:31.014618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1153/2321] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:32:31.017671 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5f99554d6ee4433b3df3e92eabae5c2', 'request_id': 'req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:31.024748 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5f99554d6ee4433b3df3e92eabae5c2', 'request_id': 'req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:31.043560 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] The token cache is being invalidate because project e5f99554d6ee4433b3df3e92eabae5c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:31.043933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:31.044629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:31.046430 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1169/2322] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:30 2022] DELETE /identity/v3/projects/e5f99554d6ee4433b3df3e92eabae5c2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:31.050697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf09e4-6a4f-4f4b-9744-cb9d119f59c6 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:31.074114 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf09e4-6a4f-4f4b-9744-cb9d119f59c6 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:31.153384 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dad13-b490-4abf-9880-9772e60037cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:31.154366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dad13-b490-4abf-9880-9772e60037cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:31.154627 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dad13-b490-4abf-9880-9772e60037cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:31.154957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dad13-b490-4abf-9880-9772e60037cc admin admin] PATH_INFO: `/v3/users/0b97696221e74f10b03e62672baf85f4` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:31.156652 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dad13-b490-4abf-9880-9772e60037cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b97696221e74f10b03e62672baf85f4)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:31.159376 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dad13-b490-4abf-9880-9772e60037cc admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:31.214547 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cf7dad13-b490-4abf-9880-9772e60037cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b97696221e74f10b03e62672baf85f4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:31.220200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:31.224130 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cf7dad13-b490-4abf-9880-9772e60037cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b97696221e74f10b03e62672baf85f4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:31.225259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cf7dad13-b490-4abf-9880-9772e60037cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b97696221e74f10b03e62672baf85f4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:31.242837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cf7dad13-b490-4abf-9880-9772e60037cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b97696221e74f10b03e62672baf85f4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:31.250741 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cf7dad13-b490-4abf-9880-9772e60037cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b97696221e74f10b03e62672baf85f4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:31.259627 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1154/2323] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:31 2022] DELETE /identity/v3/users/0b97696221e74f10b03e62672baf85f4 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:32.052098 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:32.052098 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:32.073202 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d79ac5-a5a8-4671-ba6b-42d8fb001fda admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:32.166692 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dad13-b490-4abf-9880-9772e60037cc admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:32.167653 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c63ba-e0e8-4065-89af-15a95e7b292a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:32.168064 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cf7dad13-b490-4abf-9880-9772e60037cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:32.168798 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c63ba-e0e8-4065-89af-15a95e7b292a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:32.169226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c63ba-e0e8-4065-89af-15a95e7b292a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:32.169635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c63ba-e0e8-4065-89af-15a95e7b292a admin admin] PATH_INFO: `/v3/projects/85e812d1942e44f7a2e9a25839c09153` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:32.170949 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c63ba-e0e8-4065-89af-15a95e7b292a admin admin] RBAC: Authorizing `identity:delete_project(project_id=85e812d1942e44f7a2e9a25839c09153)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:32.173229 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c63ba-e0e8-4065-89af-15a95e7b292a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:32.240994 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c36c63ba-e0e8-4065-89af-15a95e7b292a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85e812d1942e44f7a2e9a25839c09153', 'request_id': 'req-c36c63ba-e0e8-4065-89af-15a95e7b292a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:32.243589 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:32.244968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dad13-b490-4abf-9880-9772e60037cc admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:32.250092 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c36c63ba-e0e8-4065-89af-15a95e7b292a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85e812d1942e44f7a2e9a25839c09153', 'request_id': 'req-c36c63ba-e0e8-4065-89af-15a95e7b292a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:32.259564 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c36c63ba-e0e8-4065-89af-15a95e7b292a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85e812d1942e44f7a2e9a25839c09153', 'request_id': 'req-c36c63ba-e0e8-4065-89af-15a95e7b292a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:32.276122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c36c63ba-e0e8-4065-89af-15a95e7b292a admin admin] The token cache is being invalidate because project 85e812d1942e44f7a2e9a25839c09153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:32.276660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c36c63ba-e0e8-4065-89af-15a95e7b292a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:32.277532 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c36c63ba-e0e8-4065-89af-15a95e7b292a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:32.279690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1170/2324] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:32 2022] DELETE /identity/v3/projects/85e812d1942e44f7a2e9a25839c09153 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:32.287726 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3aa9d2-70dd-400c-b8aa-5b3e03106705 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:32.288149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3aa9d2-70dd-400c-b8aa-5b3e03106705 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:32.288599 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3aa9d2-70dd-400c-b8aa-5b3e03106705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:32.309147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a98b18-cc77-45b8-8c31-afbde694c1c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:32.310090 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a98b18-cc77-45b8-8c31-afbde694c1c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:32.310533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a98b18-cc77-45b8-8c31-afbde694c1c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:32.310972 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a98b18-cc77-45b8-8c31-afbde694c1c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:32.312351 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3e3aa9d2-70dd-400c-b8aa-5b3e03106705 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:32.313106 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-25a98b18-cc77-45b8-8c31-afbde694c1c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:32.314035 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3e3aa9d2-70dd-400c-b8aa-5b3e03106705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:32.388782 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a98b18-cc77-45b8-8c31-afbde694c1c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:32.390387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a98b18-cc77-45b8-8c31-afbde694c1c2 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:32.391534 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1171/2325] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:32.399610 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c47b51-4251-4cc1-af05-280cc6a579f3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:32.400000 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c47b51-4251-4cc1-af05-280cc6a579f3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:32.400377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c47b51-4251-4cc1-af05-280cc6a579f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:32.400880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1155/2326] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:32:32.428646 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-63c47b51-4251-4cc1-af05-280cc6a579f3 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:32.430250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-63c47b51-4251-4cc1-af05-280cc6a579f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:32.439783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1172/2327] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:32:32.443998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a98b18-cc77-45b8-8c31-afbde694c1c2 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:32.456247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a98b18-cc77-45b8-8c31-afbde694c1c2 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:32.461846 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47d93b-19e8-4438-98cb-4396e449f921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:32.462542 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47d93b-19e8-4438-98cb-4396e449f921 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:32.462929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47d93b-19e8-4438-98cb-4396e449f921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:32.463398 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47d93b-19e8-4438-98cb-4396e449f921 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:32.464158 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47d93b-19e8-4438-98cb-4396e449f921 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:32:32.464552 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47d93b-19e8-4438-98cb-4396e449f921 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:32.466516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47d93b-19e8-4438-98cb-4396e449f921 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:32.483594 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1156/2328] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:32:32.486521 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c47b51-4251-4cc1-af05-280cc6a579f3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:32.497856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c47b51-4251-4cc1-af05-280cc6a579f3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:32.507821 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e064a81-f41d-43ca-933d-2d2d107e4070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:32.508784 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e064a81-f41d-43ca-933d-2d2d107e4070 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:32.508784 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e064a81-f41d-43ca-933d-2d2d107e4070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:32.509032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e064a81-f41d-43ca-933d-2d2d107e4070 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:32.510317 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e064a81-f41d-43ca-933d-2d2d107e4070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:32.514733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e064a81-f41d-43ca-933d-2d2d107e4070 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:32.560769 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1173/2329] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:32 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:32:32.565412 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47d93b-19e8-4438-98cb-4396e449f921 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:32.584460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47d93b-19e8-4438-98cb-4396e449f921 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:32.642658 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:32.718925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a13e4-bdd9-42f1-8c50-43fcd8d7475c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:32.720708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a13e4-bdd9-42f1-8c50-43fcd8d7475c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:32.721154 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a13e4-bdd9-42f1-8c50-43fcd8d7475c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:32.721721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a13e4-bdd9-42f1-8c50-43fcd8d7475c admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:32.722537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a13e4-bdd9-42f1-8c50-43fcd8d7475c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:32.725497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a13e4-bdd9-42f1-8c50-43fcd8d7475c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:32.771570 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1157/2330] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:32 2022] POST /identity/v3/users => generated 292 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:32.775685 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e064a81-f41d-43ca-933d-2d2d107e4070 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:32.791391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e064a81-f41d-43ca-933d-2d2d107e4070 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:32.797061 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e37a12-7a64-4c77-83ce-9989a7442591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:32.798305 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e37a12-7a64-4c77-83ce-9989a7442591 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:32.798685 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e37a12-7a64-4c77-83ce-9989a7442591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:32.799194 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e37a12-7a64-4c77-83ce-9989a7442591 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:32.799953 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e37a12-7a64-4c77-83ce-9989a7442591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:32.801870 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e37a12-7a64-4c77-83ce-9989a7442591 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:32.815122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1174/2331] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:32.819047 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a13e4-bdd9-42f1-8c50-43fcd8d7475c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:32.833853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a13e4-bdd9-42f1-8c50-43fcd8d7475c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:32.847859 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f278eba-87f7-4226-95f1-37ba4bf7569c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:32.848912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f278eba-87f7-4226-95f1-37ba4bf7569c admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:32.849494 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f278eba-87f7-4226-95f1-37ba4bf7569c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:32.849993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f278eba-87f7-4226-95f1-37ba4bf7569c admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:32.850868 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f278eba-87f7-4226-95f1-37ba4bf7569c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:32.852741 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f278eba-87f7-4226-95f1-37ba4bf7569c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:32.873444 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1158/2332] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:32.877660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e37a12-7a64-4c77-83ce-9989a7442591 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:32.888776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e37a12-7a64-4c77-83ce-9989a7442591 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:32.894049 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f7e89-117d-4277-88b0-89b825c6cb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:32.894726 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f7e89-117d-4277-88b0-89b825c6cb14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:32.899520 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f7e89-117d-4277-88b0-89b825c6cb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:32.900148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f7e89-117d-4277-88b0-89b825c6cb14 admin admin] PATH_INFO: `/v3/projects/a089bd6924994721aabc5156ac4d83e3/users/6bc13ca4133042b7bfb92a8710b782a5/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:32.923862 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f7e89-117d-4277-88b0-89b825c6cb14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a089bd6924994721aabc5156ac4d83e3, user_id=6bc13ca4133042b7bfb92a8710b782a5, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:32.927291 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f7e89-117d-4277-88b0-89b825c6cb14 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:32.958664 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1175/2333] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:32 2022] PUT /identity/v3/projects/a089bd6924994721aabc5156ac4d83e3/users/6bc13ca4133042b7bfb92a8710b782a5/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:32.972657 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4217be-0a2f-46d5-ba8a-01520ddb2bdd None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:32.973113 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4217be-0a2f-46d5-ba8a-01520ddb2bdd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:32.973465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4217be-0a2f-46d5-ba8a-01520ddb2bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:33.026327 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1a4217be-0a2f-46d5-ba8a-01520ddb2bdd None None] MFA Rules not processed for user `6bc13ca4133042b7bfb92a8710b782a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:33.028188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1a4217be-0a2f-46d5-ba8a-01520ddb2bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:33.149862 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1159/2334] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:33.154061 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f7e89-117d-4277-88b0-89b825c6cb14 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:33.290220 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f7e89-117d-4277-88b0-89b825c6cb14 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:33.295038 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf4efb-9e86-4c18-9fd1-6547e00c2268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:33.295694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf4efb-9e86-4c18-9fd1-6547e00c2268 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:33.295972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf4efb-9e86-4c18-9fd1-6547e00c2268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:33.296244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf4efb-9e86-4c18-9fd1-6547e00c2268 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:33.296942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf4efb-9e86-4c18-9fd1-6547e00c2268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:33.300573 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf4efb-9e86-4c18-9fd1-6547e00c2268 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:33.338111 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1176/2335] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:33 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:32:33.342689 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4217be-0a2f-46d5-ba8a-01520ddb2bdd None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:33.359251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf4efb-9e86-4c18-9fd1-6547e00c2268 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:33.361921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b1bf4efb-9e86-4c18-9fd1-6547e00c2268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:33.367824 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4217be-0a2f-46d5-ba8a-01520ddb2bdd None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:33.368561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:33.440615 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf4efb-9e86-4c18-9fd1-6547e00c2268 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:33.445256 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567424e-973d-44e7-b1ad-5129ab6a96b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:33.445882 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567424e-973d-44e7-b1ad-5129ab6a96b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:33.446874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567424e-973d-44e7-b1ad-5129ab6a96b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:33.447334 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567424e-973d-44e7-b1ad-5129ab6a96b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:33.449491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3567424e-973d-44e7-b1ad-5129ab6a96b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:33.511554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba53b10-6131-4686-991c-08769a2a0442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:33.512369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba53b10-6131-4686-991c-08769a2a0442 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:33.513405 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba53b10-6131-4686-991c-08769a2a0442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:33.513690 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba53b10-6131-4686-991c-08769a2a0442 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:33.514452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba53b10-6131-4686-991c-08769a2a0442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:33.519962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba53b10-6131-4686-991c-08769a2a0442 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:33.541552 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3567424e-973d-44e7-b1ad-5129ab6a96b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:33.543186 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3567424e-973d-44e7-b1ad-5129ab6a96b4 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:33.554313 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1177/2336] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:32:33.555488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1160/2337] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:33 2022] POST /identity/v3/users => generated 299 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:33.556801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567424e-973d-44e7-b1ad-5129ab6a96b4 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:33.569503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567424e-973d-44e7-b1ad-5129ab6a96b4 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:33.582084 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d4fff-20d2-4223-8fe6-d1943f81e93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:33.582935 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d4fff-20d2-4223-8fe6-d1943f81e93c admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:33.583305 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d4fff-20d2-4223-8fe6-d1943f81e93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:33.583812 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d4fff-20d2-4223-8fe6-d1943f81e93c admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:33.584640 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d4fff-20d2-4223-8fe6-d1943f81e93c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:33.587566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d4fff-20d2-4223-8fe6-d1943f81e93c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:33.602354 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1178/2338] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:32:33.606681 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba53b10-6131-4686-991c-08769a2a0442 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:33.619337 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba53b10-6131-4686-991c-08769a2a0442 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:33.628777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1a095-277b-42ca-b708-f924ffd1bbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:33.629475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1a095-277b-42ca-b708-f924ffd1bbfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:33.629809 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1a095-277b-42ca-b708-f924ffd1bbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:33.630092 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1a095-277b-42ca-b708-f924ffd1bbfa admin admin] PATH_INFO: `/v3/projects/4fed12f373184ae59c2bead68c781d26/users/8520b3a76f824c74ae440d44aa490c7d/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:33.661426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1a095-277b-42ca-b708-f924ffd1bbfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fed12f373184ae59c2bead68c781d26, user_id=8520b3a76f824c74ae440d44aa490c7d, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:33.665985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1a095-277b-42ca-b708-f924ffd1bbfa admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:33.683889 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:33.689804 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1161/2339] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:33 2022] PUT /identity/v3/projects/4fed12f373184ae59c2bead68c781d26/users/8520b3a76f824c74ae440d44aa490c7d/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:33.693798 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d4fff-20d2-4223-8fe6-d1943f81e93c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:33.709797 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d4fff-20d2-4223-8fe6-d1943f81e93c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:33.859033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83907ddb-87f6-482b-97a4-bf7bfae040c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:33.859602 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83907ddb-87f6-482b-97a4-bf7bfae040c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:33.861588 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83907ddb-87f6-482b-97a4-bf7bfae040c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:33.861588 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83907ddb-87f6-482b-97a4-bf7bfae040c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:33.862299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83907ddb-87f6-482b-97a4-bf7bfae040c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:33.864886 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83907ddb-87f6-482b-97a4-bf7bfae040c0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:33.900109 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1179/2340] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:33.904328 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1a095-277b-42ca-b708-f924ffd1bbfa admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:33.932888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1a095-277b-42ca-b708-f924ffd1bbfa admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:33.955717 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ca264-6eea-4945-9584-fc94f1e43ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:33.956429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ca264-6eea-4945-9584-fc94f1e43ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:33.956712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ca264-6eea-4945-9584-fc94f1e43ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:33.956984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ca264-6eea-4945-9584-fc94f1e43ba5 admin admin] PATH_INFO: `/v3/projects/4fed12f373184ae59c2bead68c781d26/users/8520b3a76f824c74ae440d44aa490c7d/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:33.962959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ca264-6eea-4945-9584-fc94f1e43ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fed12f373184ae59c2bead68c781d26, user_id=8520b3a76f824c74ae440d44aa490c7d, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:33.967236 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ca264-6eea-4945-9584-fc94f1e43ba5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:33.990558 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1162/2341] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:33 2022] PUT /identity/v3/projects/4fed12f373184ae59c2bead68c781d26/users/8520b3a76f824c74ae440d44aa490c7d/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:34.000536 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d20f5-cd43-4b87-a662-adbb23dcaf7f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:34.001558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d20f5-cd43-4b87-a662-adbb23dcaf7f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:34.001861 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d20f5-cd43-4b87-a662-adbb23dcaf7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:34.064537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-cd4d20f5-cd43-4b87-a662-adbb23dcaf7f None None] MFA Rules not processed for user `8520b3a76f824c74ae440d44aa490c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:34.065839 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cd4d20f5-cd43-4b87-a662-adbb23dcaf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:34.112761 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:34.209850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1180/2342] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:34.222551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ca264-6eea-4945-9584-fc94f1e43ba5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:34.224355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-528ca264-6eea-4945-9584-fc94f1e43ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:34.311706 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ca264-6eea-4945-9584-fc94f1e43ba5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:34.317827 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e18a3-1415-4b2c-8cb9-d06eebee755c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:34.318919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e18a3-1415-4b2c-8cb9-d06eebee755c service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:34.318919 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e18a3-1415-4b2c-8cb9-d06eebee755c service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:34.319693 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e18a3-1415-4b2c-8cb9-d06eebee755c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:34.391951 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e18a3-1415-4b2c-8cb9-d06eebee755c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:34.395181 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e18a3-1415-4b2c-8cb9-d06eebee755c service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:34.405547 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1163/2343] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:34.686363 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d20f5-cd43-4b87-a662-adbb23dcaf7f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:34.694790 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cd4d20f5-cd43-4b87-a662-adbb23dcaf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:34.888270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d20f5-cd43-4b87-a662-adbb23dcaf7f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:34.896366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fc38f-5561-41ad-94ba-6c402126ee6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e935ba63cdd4256874b99a2eb0a1a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:34.897150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fc38f-5561-41ad-94ba-6c402126ee6e service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:34.897507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fc38f-5561-41ad-94ba-6c402126ee6e service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:34.897846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fc38f-5561-41ad-94ba-6c402126ee6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:34.900919 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-111fc38f-5561-41ad-94ba-6c402126ee6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:34.934250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111fc38f-5561-41ad-94ba-6c402126ee6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:34.940544 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111fc38f-5561-41ad-94ba-6c402126ee6e service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:34.957561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1181/2344] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:36.303542 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e18a3-1415-4b2c-8cb9-d06eebee755c service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:36.303542 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-898e18a3-1415-4b2c-8cb9-d06eebee755c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:36.383304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e18a3-1415-4b2c-8cb9-d06eebee755c service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:36.389506 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32a58f-7015-49d0-93c5-1255a2b174bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:36.390165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32a58f-7015-49d0-93c5-1255a2b174bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:36.390451 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32a58f-7015-49d0-93c5-1255a2b174bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:36.390725 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32a58f-7015-49d0-93c5-1255a2b174bf admin admin] PATH_INFO: `/v3/users/d9f9821cd90e4d9096cc01aee5c0e569` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:36.393106 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32a58f-7015-49d0-93c5-1255a2b174bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9f9821cd90e4d9096cc01aee5c0e569)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:36.395085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32a58f-7015-49d0-93c5-1255a2b174bf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:36.485440 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:36.495894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ba32a58f-7015-49d0-93c5-1255a2b174bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9f9821cd90e4d9096cc01aee5c0e569'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:36.507421 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ba32a58f-7015-49d0-93c5-1255a2b174bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9f9821cd90e4d9096cc01aee5c0e569'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:36.508042 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ba32a58f-7015-49d0-93c5-1255a2b174bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9f9821cd90e4d9096cc01aee5c0e569'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:36.556765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ba32a58f-7015-49d0-93c5-1255a2b174bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9f9821cd90e4d9096cc01aee5c0e569'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:36.573273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fc38f-5561-41ad-94ba-6c402126ee6e service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:36.574693 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-111fc38f-5561-41ad-94ba-6c402126ee6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:36.584022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ba32a58f-7015-49d0-93c5-1255a2b174bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9f9821cd90e4d9096cc01aee5c0e569'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:36.593166 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1164/2345] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:36 2022] DELETE /identity/v3/users/d9f9821cd90e4d9096cc01aee5c0e569 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:36.684351 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fc38f-5561-41ad-94ba-6c402126ee6e service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:36.748028 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:36.755841 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2f706-bda5-4ad5-8b47-9a5020df449c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:36.757061 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2f706-bda5-4ad5-8b47-9a5020df449c service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:36.757431 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2f706-bda5-4ad5-8b47-9a5020df449c service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:36.757799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2f706-bda5-4ad5-8b47-9a5020df449c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:36.759752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-dcd2f706-bda5-4ad5-8b47-9a5020df449c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:36.846608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2f706-bda5-4ad5-8b47-9a5020df449c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:36.848350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2f706-bda5-4ad5-8b47-9a5020df449c service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:36.860677 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1182/2346] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:32:36.861636 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32a58f-7015-49d0-93c5-1255a2b174bf admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:36.863222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ba32a58f-7015-49d0-93c5-1255a2b174bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:36.882344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32a58f-7015-49d0-93c5-1255a2b174bf admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:36.892355 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357ca2e-406a-4831-aabd-868e5edea472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:36.893216 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357ca2e-406a-4831-aabd-868e5edea472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:36.893646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357ca2e-406a-4831-aabd-868e5edea472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:36.894048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357ca2e-406a-4831-aabd-868e5edea472 admin admin] PATH_INFO: `/v3/projects/f6ff3da9d38f423ba8fbd11f635f2e1e` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:36.895869 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357ca2e-406a-4831-aabd-868e5edea472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ff3da9d38f423ba8fbd11f635f2e1e)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:36.898948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357ca2e-406a-4831-aabd-868e5edea472 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:36.973255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8357ca2e-406a-4831-aabd-868e5edea472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ff3da9d38f423ba8fbd11f635f2e1e', 'request_id': 'req-8357ca2e-406a-4831-aabd-868e5edea472'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:36.980411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8357ca2e-406a-4831-aabd-868e5edea472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ff3da9d38f423ba8fbd11f635f2e1e', 'request_id': 'req-8357ca2e-406a-4831-aabd-868e5edea472'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:36.990615 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8357ca2e-406a-4831-aabd-868e5edea472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ff3da9d38f423ba8fbd11f635f2e1e', 'request_id': 'req-8357ca2e-406a-4831-aabd-868e5edea472'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:37.013058 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8357ca2e-406a-4831-aabd-868e5edea472 admin admin] The token cache is being invalidate because project f6ff3da9d38f423ba8fbd11f635f2e1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:37.013493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8357ca2e-406a-4831-aabd-868e5edea472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:37.014278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8357ca2e-406a-4831-aabd-868e5edea472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:37.016354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1165/2347] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:36 2022] DELETE /identity/v3/projects/f6ff3da9d38f423ba8fbd11f635f2e1e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:37.022287 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c9eb-6afa-4890-8d99-ec075f45bef7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:37.022614 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c9eb-6afa-4890-8d99-ec075f45bef7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:37.022855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c9eb-6afa-4890-8d99-ec075f45bef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:37.044304 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f140c9eb-6afa-4890-8d99-ec075f45bef7 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:37.045503 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f140c9eb-6afa-4890-8d99-ec075f45bef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:37.122213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1183/2348] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:37.130800 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1d401-ed3d-418e-8cdb-66ac4807a801 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:37.131700 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1d401-ed3d-418e-8cdb-66ac4807a801 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:37.132009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1d401-ed3d-418e-8cdb-66ac4807a801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:37.152442 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e4e1d401-ed3d-418e-8cdb-66ac4807a801 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:37.153646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e4e1d401-ed3d-418e-8cdb-66ac4807a801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:37.163789 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1166/2349] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:37.167902 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140c9eb-6afa-4890-8d99-ec075f45bef7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:37.179603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140c9eb-6afa-4890-8d99-ec075f45bef7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:37.184759 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4112506-4c50-43c9-ada9-5aa0de8cbe72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:37.185384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4112506-4c50-43c9-ada9-5aa0de8cbe72 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:37.185724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4112506-4c50-43c9-ada9-5aa0de8cbe72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:37.186007 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4112506-4c50-43c9-ada9-5aa0de8cbe72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:37.186668 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4112506-4c50-43c9-ada9-5aa0de8cbe72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:32:37.186975 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4112506-4c50-43c9-ada9-5aa0de8cbe72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:37.189629 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4112506-4c50-43c9-ada9-5aa0de8cbe72 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:37.202513 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1184/2350] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:32:37.207227 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1d401-ed3d-418e-8cdb-66ac4807a801 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:37.217407 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1d401-ed3d-418e-8cdb-66ac4807a801 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:37.224996 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280122c0-e479-4ccd-9049-ae98bb84835c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:37.225983 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280122c0-e479-4ccd-9049-ae98bb84835c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:37.226452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280122c0-e479-4ccd-9049-ae98bb84835c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:37.227339 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280122c0-e479-4ccd-9049-ae98bb84835c admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:37.227993 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280122c0-e479-4ccd-9049-ae98bb84835c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:37.230080 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280122c0-e479-4ccd-9049-ae98bb84835c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:37.262555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1167/2351] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:37 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:32:37.266965 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4112506-4c50-43c9-ada9-5aa0de8cbe72 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:37.277788 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4112506-4c50-43c9-ada9-5aa0de8cbe72 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:37.341936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414d8c6-6252-491b-9641-7730fa4bcd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:37.342617 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414d8c6-6252-491b-9641-7730fa4bcd83 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:37.342891 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414d8c6-6252-491b-9641-7730fa4bcd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:37.343194 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414d8c6-6252-491b-9641-7730fa4bcd83 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:37.343835 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414d8c6-6252-491b-9641-7730fa4bcd83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:37.346439 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414d8c6-6252-491b-9641-7730fa4bcd83 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:37.370882 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1185/2352] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:37 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:37.375247 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280122c0-e479-4ccd-9049-ae98bb84835c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:37.386523 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280122c0-e479-4ccd-9049-ae98bb84835c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:37.391251 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe4b99-9a53-4816-a7b4-25914751fbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:37.391876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe4b99-9a53-4816-a7b4-25914751fbdf admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:37.392147 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe4b99-9a53-4816-a7b4-25914751fbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:37.392420 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe4b99-9a53-4816-a7b4-25914751fbdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:37.393079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe4b99-9a53-4816-a7b4-25914751fbdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:37.395175 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe4b99-9a53-4816-a7b4-25914751fbdf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:37.408844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1168/2353] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:32:37.412947 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414d8c6-6252-491b-9641-7730fa4bcd83 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:37.424685 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414d8c6-6252-491b-9641-7730fa4bcd83 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:37.429413 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fb100-c73f-47c3-bfc6-b2767b271572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:37.430092 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fb100-c73f-47c3-bfc6-b2767b271572 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:37.430439 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fb100-c73f-47c3-bfc6-b2767b271572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:37.430781 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fb100-c73f-47c3-bfc6-b2767b271572 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:37.431531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3fb100-c73f-47c3-bfc6-b2767b271572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:37.433204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3fb100-c73f-47c3-bfc6-b2767b271572 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:37.448580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1186/2354] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:37.449817 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe4b99-9a53-4816-a7b4-25914751fbdf admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:37.460820 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe4b99-9a53-4816-a7b4-25914751fbdf admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:37.466098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6be01-6a84-4319-87f3-24e08aeec7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:37.466782 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6be01-6a84-4319-87f3-24e08aeec7aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:37.467083 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6be01-6a84-4319-87f3-24e08aeec7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:37.467370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6be01-6a84-4319-87f3-24e08aeec7aa admin admin] PATH_INFO: `/v3/projects/2fb648389bd14beaae115f38b90b87e8/users/4f0cb11a5169422fb6cbb2a4c3ab64eb/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:37.484269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6be01-6a84-4319-87f3-24e08aeec7aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb648389bd14beaae115f38b90b87e8, user_id=4f0cb11a5169422fb6cbb2a4c3ab64eb, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:37.488035 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6be01-6a84-4319-87f3-24e08aeec7aa admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:37.502628 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1169/2355] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:37 2022] PUT /identity/v3/projects/2fb648389bd14beaae115f38b90b87e8/users/4f0cb11a5169422fb6cbb2a4c3ab64eb/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:37.512694 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fb926-d6d8-4974-9d78-8eba08f5f684 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:37.512985 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fb926-d6d8-4974-9d78-8eba08f5f684 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:37.513253 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fb926-d6d8-4974-9d78-8eba08f5f684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:37.522050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:37.548549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fe0fb926-d6d8-4974-9d78-8eba08f5f684 None None] MFA Rules not processed for user `4f0cb11a5169422fb6cbb2a4c3ab64eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:37.549789 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fe0fb926-d6d8-4974-9d78-8eba08f5f684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:37.620382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1187/2356] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:37.632378 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6be01-6a84-4319-87f3-24e08aeec7aa admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:37.634461 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e3b6be01-6a84-4319-87f3-24e08aeec7aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:37.690688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:37.778291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6be01-6a84-4319-87f3-24e08aeec7aa admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:37.786094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71c24e-e5e1-4579-8619-8699be08758b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:37.786823 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71c24e-e5e1-4579-8619-8699be08758b service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:37.787221 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71c24e-e5e1-4579-8619-8699be08758b service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:37.787598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71c24e-e5e1-4579-8619-8699be08758b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:37.796516 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71c24e-e5e1-4579-8619-8699be08758b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:37.798539 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71c24e-e5e1-4579-8619-8699be08758b service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:37.823305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1170/2357] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:38.158104 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fb926-d6d8-4974-9d78-8eba08f5f684 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:38.159799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fe0fb926-d6d8-4974-9d78-8eba08f5f684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:38.261330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fb926-d6d8-4974-9d78-8eba08f5f684 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:38.276127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d10af-b995-4725-8398-3623e3aa9f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:38.277270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d10af-b995-4725-8398-3623e3aa9f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:38.277270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d10af-b995-4725-8398-3623e3aa9f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:38.277270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d10af-b995-4725-8398-3623e3aa9f74 admin admin] PATH_INFO: `/v3/users/f9c17d0e2f274942abd77a90bb7d55d6` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:38.278558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d10af-b995-4725-8398-3623e3aa9f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9c17d0e2f274942abd77a90bb7d55d6)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:38.280525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d10af-b995-4725-8398-3623e3aa9f74 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:38.341595 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9e7d10af-b995-4725-8398-3623e3aa9f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9c17d0e2f274942abd77a90bb7d55d6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:38.349548 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9e7d10af-b995-4725-8398-3623e3aa9f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9c17d0e2f274942abd77a90bb7d55d6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:38.363297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9e7d10af-b995-4725-8398-3623e3aa9f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9c17d0e2f274942abd77a90bb7d55d6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:38.372031 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9e7d10af-b995-4725-8398-3623e3aa9f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9c17d0e2f274942abd77a90bb7d55d6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:38.372801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9e7d10af-b995-4725-8398-3623e3aa9f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9c17d0e2f274942abd77a90bb7d55d6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:38.379814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1188/2358] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:38 2022] DELETE /identity/v3/users/f9c17d0e2f274942abd77a90bb7d55d6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:38.384267 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a4cb3-6981-48bd-b393-56eb04f8e151 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:38.384778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a4cb3-6981-48bd-b393-56eb04f8e151 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:38.384896 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a4cb3-6981-48bd-b393-56eb04f8e151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:38.406808 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f24a4cb3-6981-48bd-b393-56eb04f8e151 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:38.407766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f24a4cb3-6981-48bd-b393-56eb04f8e151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:38.470397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1171/2359] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:32:38.480784 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d10af-b995-4725-8398-3623e3aa9f74 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:38.482518 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9e7d10af-b995-4725-8398-3623e3aa9f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:38.504196 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:38.579736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d10af-b995-4725-8398-3623e3aa9f74 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:38.585591 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60c740-8233-421c-b163-d16768bd38b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:38.587036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60c740-8233-421c-b163-d16768bd38b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:38.587636 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60c740-8233-421c-b163-d16768bd38b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:38.587872 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60c740-8233-421c-b163-d16768bd38b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:38.600390 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60c740-8233-421c-b163-d16768bd38b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:38.602239 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60c740-8233-421c-b163-d16768bd38b5 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:38.619418 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1189/2360] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:38.935012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a4cb3-6981-48bd-b393-56eb04f8e151 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:38.936812 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f24a4cb3-6981-48bd-b393-56eb04f8e151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:38.957419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a4cb3-6981-48bd-b393-56eb04f8e151 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:38.962604 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:38.964764 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:38.966388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:38.966388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] PATH_INFO: `/v3/projects/39257b45e242410ba055d41edd921288` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:38.976967 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39257b45e242410ba055d41edd921288)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:38.979882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:39.055012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39257b45e242410ba055d41edd921288', 'request_id': 'req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:39.063495 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39257b45e242410ba055d41edd921288', 'request_id': 'req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:39.072776 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39257b45e242410ba055d41edd921288', 'request_id': 'req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:39.092571 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] The token cache is being invalidate because project 39257b45e242410ba055d41edd921288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:39.092959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:39.093740 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:39.098352 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1172/2361] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:38 2022] DELETE /identity/v3/projects/39257b45e242410ba055d41edd921288 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:39.099731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60c740-8233-421c-b163-d16768bd38b5 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:39.115518 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60c740-8233-421c-b163-d16768bd38b5 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:39.219024 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3d2a2-1f13-409c-b404-a1081c014e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:39.219973 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d2a2-1f13-409c-b404-a1081c014e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:39.220328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d2a2-1f13-409c-b404-a1081c014e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:39.220675 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d2a2-1f13-409c-b404-a1081c014e77 admin admin] PATH_INFO: `/v3/users/d66b1db5a691431ea7ee2ee4e1ad1bea` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:39.221940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3d2a2-1f13-409c-b404-a1081c014e77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d66b1db5a691431ea7ee2ee4e1ad1bea)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:39.223979 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3d2a2-1f13-409c-b404-a1081c014e77 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:39.299188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:39.300884 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:39.347578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-10b3d2a2-1f13-409c-b404-a1081c014e77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd66b1db5a691431ea7ee2ee4e1ad1bea'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:39.349986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:39.378454 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-10b3d2a2-1f13-409c-b404-a1081c014e77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd66b1db5a691431ea7ee2ee4e1ad1bea'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:39.399559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-10b3d2a2-1f13-409c-b404-a1081c014e77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd66b1db5a691431ea7ee2ee4e1ad1bea'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:39.408697 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-10b3d2a2-1f13-409c-b404-a1081c014e77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd66b1db5a691431ea7ee2ee4e1ad1bea'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:39.409543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-10b3d2a2-1f13-409c-b404-a1081c014e77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd66b1db5a691431ea7ee2ee4e1ad1bea'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:39.424228 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1190/2362] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:39 2022] DELETE /identity/v3/users/d66b1db5a691431ea7ee2ee4e1ad1bea => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:39.438894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564c99a-be44-4196-ab69-2fe9a6ff8ab2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:39.503448 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefa141-14d7-4f2b-94f0-010113e24ce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:39.504174 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefa141-14d7-4f2b-94f0-010113e24ce1 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:39.504543 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefa141-14d7-4f2b-94f0-010113e24ce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:39.505095 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefa141-14d7-4f2b-94f0-010113e24ce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:39.507408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-eaefa141-14d7-4f2b-94f0-010113e24ce1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:39.607078 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefa141-14d7-4f2b-94f0-010113e24ce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:39.609045 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefa141-14d7-4f2b-94f0-010113e24ce1 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:39.634479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1173/2363] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:40.044505 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3d2a2-1f13-409c-b404-a1081c014e77 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:40.046830 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-10b3d2a2-1f13-409c-b404-a1081c014e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:40.104204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:40.139610 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3d2a2-1f13-409c-b404-a1081c014e77 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:40.148263 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:40.149161 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:40.149592 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:40.149976 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] PATH_INFO: `/v3/projects/21aadf53c1424237b34422bfcd93f50f` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:40.151569 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] RBAC: Authorizing `identity:delete_project(project_id=21aadf53c1424237b34422bfcd93f50f)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:40.153604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:40.223950 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21aadf53c1424237b34422bfcd93f50f', 'request_id': 'req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:40.232832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21aadf53c1424237b34422bfcd93f50f', 'request_id': 'req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:40.243023 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21aadf53c1424237b34422bfcd93f50f', 'request_id': 'req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:40.256898 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] The token cache is being invalidate because project 21aadf53c1424237b34422bfcd93f50f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:40.257294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:40.258021 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:40.259796 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1191/2364] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:40 2022] DELETE /identity/v3/projects/21aadf53c1424237b34422bfcd93f50f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:40.264004 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefa141-14d7-4f2b-94f0-010113e24ce1 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:40.287171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefa141-14d7-4f2b-94f0-010113e24ce1 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:40.384051 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d8eef-4336-4751-8de3-8d721dcc7d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:40.386638 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d8eef-4336-4751-8de3-8d721dcc7d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:40.386918 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d8eef-4336-4751-8de3-8d721dcc7d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:40.387303 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d8eef-4336-4751-8de3-8d721dcc7d56 admin admin] PATH_INFO: `/v3/users/3f35a912286744d591df162545b9f578` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:40.389840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d8eef-4336-4751-8de3-8d721dcc7d56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f35a912286744d591df162545b9f578)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:40.392047 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d8eef-4336-4751-8de3-8d721dcc7d56 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:40.438329 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:40.467423 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c76d8eef-4336-4751-8de3-8d721dcc7d56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f35a912286744d591df162545b9f578'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:40.476454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c76d8eef-4336-4751-8de3-8d721dcc7d56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f35a912286744d591df162545b9f578'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:40.477117 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c76d8eef-4336-4751-8de3-8d721dcc7d56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f35a912286744d591df162545b9f578'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:40.496806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c76d8eef-4336-4751-8de3-8d721dcc7d56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f35a912286744d591df162545b9f578'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:40.507410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c76d8eef-4336-4751-8de3-8d721dcc7d56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f35a912286744d591df162545b9f578'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:40.515822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1174/2365] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:40 2022] DELETE /identity/v3/users/3f35a912286744d591df162545b9f578 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:41.022145 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:41.023921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:41.038082 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1608af-19d6-41b0-8c5c-e7e8e4a7f91e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:41.115712 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76c210-5516-4a2e-b214-f8ce050c430a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:41.116396 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76c210-5516-4a2e-b214-f8ce050c430a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:41.116600 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76c210-5516-4a2e-b214-f8ce050c430a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:41.116867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76c210-5516-4a2e-b214-f8ce050c430a admin admin] PATH_INFO: `/v3/projects/45d6cfc1e16b4faba4ffe8509f45cb66` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:41.118138 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76c210-5516-4a2e-b214-f8ce050c430a admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d6cfc1e16b4faba4ffe8509f45cb66)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:41.120202 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76c210-5516-4a2e-b214-f8ce050c430a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:41.184190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7c76c210-5516-4a2e-b214-f8ce050c430a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d6cfc1e16b4faba4ffe8509f45cb66', 'request_id': 'req-7c76c210-5516-4a2e-b214-f8ce050c430a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:41.197415 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7c76c210-5516-4a2e-b214-f8ce050c430a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d6cfc1e16b4faba4ffe8509f45cb66', 'request_id': 'req-7c76c210-5516-4a2e-b214-f8ce050c430a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:41.204586 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7c76c210-5516-4a2e-b214-f8ce050c430a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d6cfc1e16b4faba4ffe8509f45cb66', 'request_id': 'req-7c76c210-5516-4a2e-b214-f8ce050c430a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:41.208453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:41.218672 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7c76c210-5516-4a2e-b214-f8ce050c430a admin admin] The token cache is being invalidate because project 45d6cfc1e16b4faba4ffe8509f45cb66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:41.219109 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7c76c210-5516-4a2e-b214-f8ce050c430a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:41.220185 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7c76c210-5516-4a2e-b214-f8ce050c430a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:41.222050 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1192/2366] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:41 2022] DELETE /identity/v3/projects/45d6cfc1e16b4faba4ffe8509f45cb66 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:41.226935 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c1085-66cb-467f-953d-2cb32ce32ead None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:41.227257 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c1085-66cb-467f-953d-2cb32ce32ead None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:41.227537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c1085-66cb-467f-953d-2cb32ce32ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:41.247666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-fe7c1085-66cb-467f-953d-2cb32ce32ead None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:41.249013 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-fe7c1085-66cb-467f-953d-2cb32ce32ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:41.308959 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1175/2367] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:41.318693 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967e380-daea-4979-9ab7-45767a6cc059 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:41.318973 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967e380-daea-4979-9ab7-45767a6cc059 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:41.319278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967e380-daea-4979-9ab7-45767a6cc059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:41.339886 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2967e380-daea-4979-9ab7-45767a6cc059 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:41.341090 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2967e380-daea-4979-9ab7-45767a6cc059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:41.349457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1193/2368] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:41.353739 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c1085-66cb-467f-953d-2cb32ce32ead None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:41.363892 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c1085-66cb-467f-953d-2cb32ce32ead None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:41.368960 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20236d0-422e-4f89-94d8-8c0ab5c260d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:41.369578 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20236d0-422e-4f89-94d8-8c0ab5c260d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:41.369853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20236d0-422e-4f89-94d8-8c0ab5c260d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:41.370122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20236d0-422e-4f89-94d8-8c0ab5c260d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:41.370767 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20236d0-422e-4f89-94d8-8c0ab5c260d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:32:41.371097 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20236d0-422e-4f89-94d8-8c0ab5c260d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:41.372747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20236d0-422e-4f89-94d8-8c0ab5c260d4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:41.389275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1176/2369] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:32:41.390783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967e380-daea-4979-9ab7-45767a6cc059 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:41.401005 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967e380-daea-4979-9ab7-45767a6cc059 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:41.405598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28f7bc-5e7e-4d59-beca-adce5206714c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:41.406200 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28f7bc-5e7e-4d59-beca-adce5206714c admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:41.406976 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28f7bc-5e7e-4d59-beca-adce5206714c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:41.407272 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28f7bc-5e7e-4d59-beca-adce5206714c admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:41.407941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28f7bc-5e7e-4d59-beca-adce5206714c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:41.409839 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28f7bc-5e7e-4d59-beca-adce5206714c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:41.442189 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1194/2370] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:41 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:32:41.443581 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20236d0-422e-4f89-94d8-8c0ab5c260d4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:41.454258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20236d0-422e-4f89-94d8-8c0ab5c260d4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:41.516089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70426883-c5ca-4b22-b229-a3c5c8c1374a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:41.516867 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70426883-c5ca-4b22-b229-a3c5c8c1374a admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:41.517032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70426883-c5ca-4b22-b229-a3c5c8c1374a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:41.517299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70426883-c5ca-4b22-b229-a3c5c8c1374a admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:41.517946 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70426883-c5ca-4b22-b229-a3c5c8c1374a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:41.519886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70426883-c5ca-4b22-b229-a3c5c8c1374a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:41.545003 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1177/2371] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:41 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:41.549724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28f7bc-5e7e-4d59-beca-adce5206714c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:41.559982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28f7bc-5e7e-4d59-beca-adce5206714c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:41.565177 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d96f0-36df-42a6-98e0-791256a3b451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:41.565783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d96f0-36df-42a6-98e0-791256a3b451 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:41.566342 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d96f0-36df-42a6-98e0-791256a3b451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:41.566617 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d96f0-36df-42a6-98e0-791256a3b451 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:41.567285 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d96f0-36df-42a6-98e0-791256a3b451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:41.568916 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d96f0-36df-42a6-98e0-791256a3b451 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:41.579887 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1195/2372] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:32:41.583985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70426883-c5ca-4b22-b229-a3c5c8c1374a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:41.594480 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70426883-c5ca-4b22-b229-a3c5c8c1374a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:41.599322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5401e6-691a-45d4-919d-3d39cec0facf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:41.600059 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5401e6-691a-45d4-919d-3d39cec0facf admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:41.600337 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5401e6-691a-45d4-919d-3d39cec0facf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:41.600607 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5401e6-691a-45d4-919d-3d39cec0facf admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:41.601270 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5401e6-691a-45d4-919d-3d39cec0facf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:41.602856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5401e6-691a-45d4-919d-3d39cec0facf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:41.616280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1178/2373] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:41.618813 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d96f0-36df-42a6-98e0-791256a3b451 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:41.629479 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d96f0-36df-42a6-98e0-791256a3b451 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:41.634352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cc97c-3735-4fa3-b7a1-f296b65ee2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:41.635012 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cc97c-3735-4fa3-b7a1-f296b65ee2e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:41.635323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cc97c-3735-4fa3-b7a1-f296b65ee2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:41.635596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cc97c-3735-4fa3-b7a1-f296b65ee2e1 admin admin] PATH_INFO: `/v3/projects/336f450089ad4487b7d0646391f6fdf5/users/bd8bba63a2f04fa78a4794fe7fc1079b/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:41.650352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55cc97c-3735-4fa3-b7a1-f296b65ee2e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=336f450089ad4487b7d0646391f6fdf5, user_id=bd8bba63a2f04fa78a4794fe7fc1079b, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:41.653044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55cc97c-3735-4fa3-b7a1-f296b65ee2e1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:41.665889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1196/2374] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:41 2022] PUT /identity/v3/projects/336f450089ad4487b7d0646391f6fdf5/users/bd8bba63a2f04fa78a4794fe7fc1079b/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:41.677379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9217dc-2b81-401b-a0af-0e89a3f687a2 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:41.677914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9217dc-2b81-401b-a0af-0e89a3f687a2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:41.678322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9217dc-2b81-401b-a0af-0e89a3f687a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:41.703965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:41.707461 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-3c9217dc-2b81-401b-a0af-0e89a3f687a2 None None] MFA Rules not processed for user `bd8bba63a2f04fa78a4794fe7fc1079b`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:41.708986 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3c9217dc-2b81-401b-a0af-0e89a3f687a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:41.786420 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1179/2375] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:41.790919 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cc97c-3735-4fa3-b7a1-f296b65ee2e1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:41.854091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cc97c-3735-4fa3-b7a1-f296b65ee2e1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:41.858912 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620e112-2580-4055-b26f-660bdef2eea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:41.860314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620e112-2580-4055-b26f-660bdef2eea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:41.860457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620e112-2580-4055-b26f-660bdef2eea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:41.860866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620e112-2580-4055-b26f-660bdef2eea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:41.861578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620e112-2580-4055-b26f-660bdef2eea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:41.863687 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620e112-2580-4055-b26f-660bdef2eea6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:41.899134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1197/2376] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:41 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:41.903054 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9217dc-2b81-401b-a0af-0e89a3f687a2 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:41.915930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9217dc-2b81-401b-a0af-0e89a3f687a2 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:41.993957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e446218-3b5c-43bd-9ad8-cbbf998c3af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:41.994922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e446218-3b5c-43bd-9ad8-cbbf998c3af1 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:41.995236 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e446218-3b5c-43bd-9ad8-cbbf998c3af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:41.995513 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e446218-3b5c-43bd-9ad8-cbbf998c3af1 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:41.996537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e446218-3b5c-43bd-9ad8-cbbf998c3af1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:41.999426 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e446218-3b5c-43bd-9ad8-cbbf998c3af1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:42.038141 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1180/2377] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:41 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:42.041759 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620e112-2580-4055-b26f-660bdef2eea6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:42.054689 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620e112-2580-4055-b26f-660bdef2eea6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:42.062812 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8f06c-350c-4b18-8512-bd3154228816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:42.063809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8f06c-350c-4b18-8512-bd3154228816 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:42.064251 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8f06c-350c-4b18-8512-bd3154228816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:42.064680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8f06c-350c-4b18-8512-bd3154228816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:42.065501 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8f06c-350c-4b18-8512-bd3154228816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:42.067553 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8f06c-350c-4b18-8512-bd3154228816 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:42.083493 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1198/2378] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:42.087182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e446218-3b5c-43bd-9ad8-cbbf998c3af1 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:42.105049 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e446218-3b5c-43bd-9ad8-cbbf998c3af1 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:42.114738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be52517-f774-4022-9637-8d12e671fa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:42.117199 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be52517-f774-4022-9637-8d12e671fa95 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:42.117698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be52517-f774-4022-9637-8d12e671fa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:42.117980 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be52517-f774-4022-9637-8d12e671fa95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:42.118819 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be52517-f774-4022-9637-8d12e671fa95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:42.120517 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be52517-f774-4022-9637-8d12e671fa95 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:42.132886 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1181/2379] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:32:42.136727 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8f06c-350c-4b18-8512-bd3154228816 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:42.149487 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8f06c-350c-4b18-8512-bd3154228816 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:42.157149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae0848-3bb2-489a-99f2-40da1347e0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:42.157918 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae0848-3bb2-489a-99f2-40da1347e0f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:42.158303 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae0848-3bb2-489a-99f2-40da1347e0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:42.158680 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae0848-3bb2-489a-99f2-40da1347e0f4 admin admin] PATH_INFO: `/v3/projects/7622ec08a8064bc6ba542b330de09b24/users/993a936a9c5c4fa7aaf7a8b151ade1ce/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:42.179583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae0848-3bb2-489a-99f2-40da1347e0f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7622ec08a8064bc6ba542b330de09b24, user_id=993a936a9c5c4fa7aaf7a8b151ade1ce, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:42.186157 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae0848-3bb2-489a-99f2-40da1347e0f4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:42.206921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1199/2380] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:42 2022] PUT /identity/v3/projects/7622ec08a8064bc6ba542b330de09b24/users/993a936a9c5c4fa7aaf7a8b151ade1ce/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:42.217484 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611b236-f1b5-4cfb-95d2-911da6b26d62 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:42.217908 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611b236-f1b5-4cfb-95d2-911da6b26d62 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:42.218297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611b236-f1b5-4cfb-95d2-911da6b26d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:42.284364 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c611b236-f1b5-4cfb-95d2-911da6b26d62 None None] MFA Rules not processed for user `993a936a9c5c4fa7aaf7a8b151ade1ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:42.286486 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c611b236-f1b5-4cfb-95d2-911da6b26d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:42.382678 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1182/2381] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:32:42.387496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae0848-3bb2-489a-99f2-40da1347e0f4 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:42.442874 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:42.466391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae0848-3bb2-489a-99f2-40da1347e0f4 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:42.472923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89a432-f7c3-4869-9919-e655e5c886d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:42.473750 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89a432-f7c3-4869-9919-e655e5c886d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:42.475040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89a432-f7c3-4869-9919-e655e5c886d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:42.475482 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89a432-f7c3-4869-9919-e655e5c886d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:42.476376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89a432-f7c3-4869-9919-e655e5c886d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:42.479320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89a432-f7c3-4869-9919-e655e5c886d0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:42.514507 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1200/2382] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:42 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:42.519519 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611b236-f1b5-4cfb-95d2-911da6b26d62 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:42.530699 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611b236-f1b5-4cfb-95d2-911da6b26d62 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:42.542957 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:42.605019 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a35e7d-e6b2-4661-8b5b-67765478bde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:42.606308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a35e7d-e6b2-4661-8b5b-67765478bde0 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:42.606747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a35e7d-e6b2-4661-8b5b-67765478bde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:42.607651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a35e7d-e6b2-4661-8b5b-67765478bde0 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:42.608429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a35e7d-e6b2-4661-8b5b-67765478bde0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:42.612380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a35e7d-e6b2-4661-8b5b-67765478bde0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:42.642891 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1183/2383] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:42 2022] POST /identity/v3/users => generated 298 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:42.643547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89a432-f7c3-4869-9919-e655e5c886d0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:42.654188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89a432-f7c3-4869-9919-e655e5c886d0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:42.665044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4565fa-c423-40d8-911d-811fd392725c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:42.665953 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4565fa-c423-40d8-911d-811fd392725c admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:42.666404 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4565fa-c423-40d8-911d-811fd392725c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:42.666837 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4565fa-c423-40d8-911d-811fd392725c admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:42.667831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4565fa-c423-40d8-911d-811fd392725c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:42.669604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4565fa-c423-40d8-911d-811fd392725c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:42.683128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1201/2384] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:32:42.688103 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a35e7d-e6b2-4661-8b5b-67765478bde0 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:42.699165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a35e7d-e6b2-4661-8b5b-67765478bde0 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:42.704737 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9b501-7ea5-42b2-b59e-24dedf65b16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:42.705423 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9b501-7ea5-42b2-b59e-24dedf65b16f admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:42.705696 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9b501-7ea5-42b2-b59e-24dedf65b16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:42.705968 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9b501-7ea5-42b2-b59e-24dedf65b16f admin admin] PATH_INFO: `/v3/projects/4a01e1dab6644e1797112dad1df6480f/users/60e9408fa6114130bc33255c9d94e625/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:42.739461 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9b501-7ea5-42b2-b59e-24dedf65b16f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a01e1dab6644e1797112dad1df6480f, user_id=60e9408fa6114130bc33255c9d94e625, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:42.743252 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9b501-7ea5-42b2-b59e-24dedf65b16f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:42.758911 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1184/2385] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:42 2022] PUT /identity/v3/projects/4a01e1dab6644e1797112dad1df6480f/users/60e9408fa6114130bc33255c9d94e625/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:42.763203 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4565fa-c423-40d8-911d-811fd392725c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:42.776048 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4565fa-c423-40d8-911d-811fd392725c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:42.852491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbc1a7-1b24-472a-a412-568e43171f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:42.853267 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbc1a7-1b24-472a-a412-568e43171f71 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:42.853552 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbc1a7-1b24-472a-a412-568e43171f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:42.853826 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbc1a7-1b24-472a-a412-568e43171f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:42.854492 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbc1a7-1b24-472a-a412-568e43171f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:42.856544 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbc1a7-1b24-472a-a412-568e43171f71 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:42.873134 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1202/2386] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:42.875177 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9b501-7ea5-42b2-b59e-24dedf65b16f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:42.889755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9b501-7ea5-42b2-b59e-24dedf65b16f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:42.895438 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff3b74-d5c2-42e0-9c4c-50ed3afe986a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:42.896087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff3b74-d5c2-42e0-9c4c-50ed3afe986a admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:42.896568 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff3b74-d5c2-42e0-9c4c-50ed3afe986a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:42.896971 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff3b74-d5c2-42e0-9c4c-50ed3afe986a admin admin] PATH_INFO: `/v3/projects/4a01e1dab6644e1797112dad1df6480f/users/60e9408fa6114130bc33255c9d94e625/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:42.899918 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff3b74-d5c2-42e0-9c4c-50ed3afe986a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a01e1dab6644e1797112dad1df6480f, user_id=60e9408fa6114130bc33255c9d94e625, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:42.903527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff3b74-d5c2-42e0-9c4c-50ed3afe986a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:42.927938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1185/2387] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:42 2022] PUT /identity/v3/projects/4a01e1dab6644e1797112dad1df6480f/users/60e9408fa6114130bc33255c9d94e625/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:42.932772 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe2b8b-0fa3-424c-a3fd-9acac82797b4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:42.933057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe2b8b-0fa3-424c-a3fd-9acac82797b4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:42.933327 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe2b8b-0fa3-424c-a3fd-9acac82797b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:42.970261 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-23fe2b8b-0fa3-424c-a3fd-9acac82797b4 None None] MFA Rules not processed for user `60e9408fa6114130bc33255c9d94e625`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:42.971549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-23fe2b8b-0fa3-424c-a3fd-9acac82797b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:43.066594 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1203/2388] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:43.081622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff3b74-d5c2-42e0-9c4c-50ed3afe986a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:43.083506 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-96ff3b74-d5c2-42e0-9c4c-50ed3afe986a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:43.204856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff3b74-d5c2-42e0-9c4c-50ed3afe986a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:43.213654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f364d8-60ec-46e6-b9a6-b8e30ad413bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:43.214600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f364d8-60ec-46e6-b9a6-b8e30ad413bc service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:43.215006 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f364d8-60ec-46e6-b9a6-b8e30ad413bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:43.215483 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f364d8-60ec-46e6-b9a6-b8e30ad413bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:43.310846 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f364d8-60ec-46e6-b9a6-b8e30ad413bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:43.313259 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f364d8-60ec-46e6-b9a6-b8e30ad413bc service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:43.329101 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1186/2389] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:44.497437 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe2b8b-0fa3-424c-a3fd-9acac82797b4 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:44.498934 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-23fe2b8b-0fa3-424c-a3fd-9acac82797b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:44.544406 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:44.571960 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe2b8b-0fa3-424c-a3fd-9acac82797b4 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:44.577116 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956c049-349e-4a84-b777-2c65d10808da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:44.577886 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956c049-349e-4a84-b777-2c65d10808da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:44.578343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956c049-349e-4a84-b777-2c65d10808da admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:44.578720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956c049-349e-4a84-b777-2c65d10808da admin admin] PATH_INFO: `/v3/users/d40a56908e934174b1f7b20e62e2cfcd` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:44.580002 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956c049-349e-4a84-b777-2c65d10808da admin admin] RBAC: Authorizing `identity:delete_user(user_id=d40a56908e934174b1f7b20e62e2cfcd)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:44.582341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956c049-349e-4a84-b777-2c65d10808da admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:44.627611 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2956c049-349e-4a84-b777-2c65d10808da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd40a56908e934174b1f7b20e62e2cfcd'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:44.634451 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2956c049-349e-4a84-b777-2c65d10808da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd40a56908e934174b1f7b20e62e2cfcd'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:44.646308 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2956c049-349e-4a84-b777-2c65d10808da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd40a56908e934174b1f7b20e62e2cfcd'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:44.652937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2956c049-349e-4a84-b777-2c65d10808da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd40a56908e934174b1f7b20e62e2cfcd'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:44.653571 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2956c049-349e-4a84-b777-2c65d10808da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd40a56908e934174b1f7b20e62e2cfcd'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:44.659441 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1204/2390] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:44 2022] DELETE /identity/v3/users/d40a56908e934174b1f7b20e62e2cfcd => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:44.862260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f364d8-60ec-46e6-b9a6-b8e30ad413bc service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:44.875294 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f364d8-60ec-46e6-b9a6-b8e30ad413bc service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:44.889655 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:44.942187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:44.943079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:44.943371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:44.943653 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] PATH_INFO: `/v3/projects/f2a51798d7e54605a7e59b0cba385973` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:44.944775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2a51798d7e54605a7e59b0cba385973)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:44.946922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:44.963312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956c049-349e-4a84-b777-2c65d10808da admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:44.964679 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2956c049-349e-4a84-b777-2c65d10808da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:45.002755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2a51798d7e54605a7e59b0cba385973', 'request_id': 'req-167b0d0a-6a8f-406b-84f5-690647bc509d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.008373 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2a51798d7e54605a7e59b0cba385973', 'request_id': 'req-167b0d0a-6a8f-406b-84f5-690647bc509d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.014502 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2a51798d7e54605a7e59b0cba385973', 'request_id': 'req-167b0d0a-6a8f-406b-84f5-690647bc509d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.026558 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] The token cache is being invalidate because project f2a51798d7e54605a7e59b0cba385973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:45.026842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.027237 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956c049-349e-4a84-b777-2c65d10808da admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:45.027482 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.028966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1187/2391] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:44 2022] DELETE /identity/v3/projects/f2a51798d7e54605a7e59b0cba385973 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:45.033286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:45.034747 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:45.079663 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c3165-bb8c-44a9-b78c-d9bec80a6000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:45.080290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c3165-bb8c-44a9-b78c-d9bec80a6000 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:45.080603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c3165-bb8c-44a9-b78c-d9bec80a6000 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:45.080880 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c3165-bb8c-44a9-b78c-d9bec80a6000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:45.082699 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-498c3165-bb8c-44a9-b78c-d9bec80a6000 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:45.100609 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b0d0a-6a8f-406b-84f5-690647bc509d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:45.105403 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a151090-848f-4b4c-a95d-9e05b31fda3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:45.106286 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a151090-848f-4b4c-a95d-9e05b31fda3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:45.106562 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a151090-848f-4b4c-a95d-9e05b31fda3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:45.106834 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a151090-848f-4b4c-a95d-9e05b31fda3d admin admin] PATH_INFO: `/v3/users/e7e25970bb534ea39821f9c9640dcb66` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:45.108043 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a151090-848f-4b4c-a95d-9e05b31fda3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e25970bb534ea39821f9c9640dcb66)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:45.110732 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a151090-848f-4b4c-a95d-9e05b31fda3d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:45.112181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:45.149764 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c3165-bb8c-44a9-b78c-d9bec80a6000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:45.151289 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c3165-bb8c-44a9-b78c-d9bec80a6000 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:45.160116 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3a151090-848f-4b4c-a95d-9e05b31fda3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e25970bb534ea39821f9c9640dcb66'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.161280 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1205/2392] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:45.162716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:45.168554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3a151090-848f-4b4c-a95d-9e05b31fda3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e25970bb534ea39821f9c9640dcb66'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.169323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3a151090-848f-4b4c-a95d-9e05b31fda3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e25970bb534ea39821f9c9640dcb66'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.184479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3a151090-848f-4b4c-a95d-9e05b31fda3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e25970bb534ea39821f9c9640dcb66'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.192234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3a151090-848f-4b4c-a95d-9e05b31fda3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e25970bb534ea39821f9c9640dcb66'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.198288 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1188/2393] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:45 2022] DELETE /identity/v3/users/e7e25970bb534ea39821f9c9640dcb66 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:45.388228 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c3165-bb8c-44a9-b78c-d9bec80a6000 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:45.402429 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c3165-bb8c-44a9-b78c-d9bec80a6000 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:45.494828 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:45.495867 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:45.496029 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:45.496310 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] PATH_INFO: `/v3/projects/a85ca09a00f045af8bccae7383271502` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:45.500722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a85ca09a00f045af8bccae7383271502)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:45.500722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:45.576467 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a85ca09a00f045af8bccae7383271502', 'request_id': 'req-f47e93ff-f1d0-4865-b9e6-82bea767320e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.583947 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a85ca09a00f045af8bccae7383271502', 'request_id': 'req-f47e93ff-f1d0-4865-b9e6-82bea767320e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.590623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a85ca09a00f045af8bccae7383271502', 'request_id': 'req-f47e93ff-f1d0-4865-b9e6-82bea767320e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.606409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] The token cache is being invalidate because project a85ca09a00f045af8bccae7383271502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:45.606697 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.610297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.614873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1206/2394] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:45 2022] DELETE /identity/v3/projects/a85ca09a00f045af8bccae7383271502 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:45.615922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a151090-848f-4b4c-a95d-9e05b31fda3d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:45.616095 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3a151090-848f-4b4c-a95d-9e05b31fda3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:45.631363 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a151090-848f-4b4c-a95d-9e05b31fda3d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:45.710367 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e4ab5-e7aa-440f-9f7e-bfd6dc58fccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:45.711025 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e4ab5-e7aa-440f-9f7e-bfd6dc58fccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:45.711400 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e4ab5-e7aa-440f-9f7e-bfd6dc58fccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:45.711675 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e4ab5-e7aa-440f-9f7e-bfd6dc58fccb admin admin] PATH_INFO: `/v3/users/4f0cb11a5169422fb6cbb2a4c3ab64eb` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:45.712873 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e4ab5-e7aa-440f-9f7e-bfd6dc58fccb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f0cb11a5169422fb6cbb2a4c3ab64eb)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:45.722038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e4ab5-e7aa-440f-9f7e-bfd6dc58fccb admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:45.765151 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:45.767202 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:45.790803 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b95e4ab5-e7aa-440f-9f7e-bfd6dc58fccb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f0cb11a5169422fb6cbb2a4c3ab64eb'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.801054 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b95e4ab5-e7aa-440f-9f7e-bfd6dc58fccb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f0cb11a5169422fb6cbb2a4c3ab64eb'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.802012 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b95e4ab5-e7aa-440f-9f7e-bfd6dc58fccb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f0cb11a5169422fb6cbb2a4c3ab64eb'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.816842 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b95e4ab5-e7aa-440f-9f7e-bfd6dc58fccb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f0cb11a5169422fb6cbb2a4c3ab64eb'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.826084 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b95e4ab5-e7aa-440f-9f7e-bfd6dc58fccb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f0cb11a5169422fb6cbb2a4c3ab64eb'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:45.833697 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1189/2395] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:45 2022] DELETE /identity/v3/users/4f0cb11a5169422fb6cbb2a4c3ab64eb => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:45.838197 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9444c29-9079-4c3c-b3de-08f3d370ca99 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:45.838517 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9444c29-9079-4c3c-b3de-08f3d370ca99 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:45.838821 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9444c29-9079-4c3c-b3de-08f3d370ca99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:45.839939 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e93ff-f1d0-4865-b9e6-82bea767320e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:45.852435 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:45.871713 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a9444c29-9079-4c3c-b3de-08f3d370ca99 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:45.873899 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a9444c29-9079-4c3c-b3de-08f3d370ca99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:45.904659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446030b8-3f1a-4739-98e2-2ffff53744c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:45.905275 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446030b8-3f1a-4739-98e2-2ffff53744c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:45.905727 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446030b8-3f1a-4739-98e2-2ffff53744c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:45.906509 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446030b8-3f1a-4739-98e2-2ffff53744c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:45.908432 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-446030b8-3f1a-4739-98e2-2ffff53744c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:45.932557 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:45.969863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1190/2396] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:32:45.980283 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9444c29-9079-4c3c-b3de-08f3d370ca99 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:45.984948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a9444c29-9079-4c3c-b3de-08f3d370ca99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:45.992449 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446030b8-3f1a-4739-98e2-2ffff53744c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:45.994505 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446030b8-3f1a-4739-98e2-2ffff53744c9 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:46.008126 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1207/2397] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:46.088223 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9444c29-9079-4c3c-b3de-08f3d370ca99 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:46.092911 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752b306-2d91-4232-b4b2-391937458c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:46.093670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752b306-2d91-4232-b4b2-391937458c39 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:46.093965 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752b306-2d91-4232-b4b2-391937458c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:46.094341 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752b306-2d91-4232-b4b2-391937458c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:46.104263 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752b306-2d91-4232-b4b2-391937458c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:46.106006 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752b306-2d91-4232-b4b2-391937458c39 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:46.116904 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1191/2398] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:46.402954 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446030b8-3f1a-4739-98e2-2ffff53744c9 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:46.404489 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-446030b8-3f1a-4739-98e2-2ffff53744c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:46.418948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446030b8-3f1a-4739-98e2-2ffff53744c9 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:46.424290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239fb94-394c-4891-a965-bdf77220eb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:46.424927 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239fb94-394c-4891-a965-bdf77220eb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:46.425363 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239fb94-394c-4891-a965-bdf77220eb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:46.425635 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239fb94-394c-4891-a965-bdf77220eb9f admin admin] PATH_INFO: `/v3/projects/2fb648389bd14beaae115f38b90b87e8` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:46.426762 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239fb94-394c-4891-a965-bdf77220eb9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb648389bd14beaae115f38b90b87e8)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:46.428716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239fb94-394c-4891-a965-bdf77220eb9f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:46.475344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5239fb94-394c-4891-a965-bdf77220eb9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb648389bd14beaae115f38b90b87e8', 'request_id': 'req-5239fb94-394c-4891-a965-bdf77220eb9f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:46.480690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5239fb94-394c-4891-a965-bdf77220eb9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb648389bd14beaae115f38b90b87e8', 'request_id': 'req-5239fb94-394c-4891-a965-bdf77220eb9f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:46.487531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5239fb94-394c-4891-a965-bdf77220eb9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb648389bd14beaae115f38b90b87e8', 'request_id': 'req-5239fb94-394c-4891-a965-bdf77220eb9f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:46.500498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5239fb94-394c-4891-a965-bdf77220eb9f admin admin] The token cache is being invalidate because project 2fb648389bd14beaae115f38b90b87e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:46.500777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5239fb94-394c-4891-a965-bdf77220eb9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:46.501393 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5239fb94-394c-4891-a965-bdf77220eb9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:46.502852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1208/2399] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:46 2022] DELETE /identity/v3/projects/2fb648389bd14beaae115f38b90b87e8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:46.507710 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13178dff-c758-425a-aa86-af793c114682 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:46.507995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13178dff-c758-425a-aa86-af793c114682 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:46.508326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13178dff-c758-425a-aa86-af793c114682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:46.528927 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-13178dff-c758-425a-aa86-af793c114682 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:46.530127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-13178dff-c758-425a-aa86-af793c114682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:46.593398 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1192/2400] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:46.601626 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafc6ee-1b54-49e9-89d2-022010e55836 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:46.601915 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafc6ee-1b54-49e9-89d2-022010e55836 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:46.602222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafc6ee-1b54-49e9-89d2-022010e55836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:46.622879 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bcafc6ee-1b54-49e9-89d2-022010e55836 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:46.624133 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bcafc6ee-1b54-49e9-89d2-022010e55836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:46.632144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1209/2401] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:46.636187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13178dff-c758-425a-aa86-af793c114682 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:46.645861 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13178dff-c758-425a-aa86-af793c114682 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:46.650354 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60766d-14ca-45ae-9cd8-34ae39115c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:46.650948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60766d-14ca-45ae-9cd8-34ae39115c0e admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:46.651281 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60766d-14ca-45ae-9cd8-34ae39115c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:46.651555 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60766d-14ca-45ae-9cd8-34ae39115c0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:46.652200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60766d-14ca-45ae-9cd8-34ae39115c0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:32:46.652499 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60766d-14ca-45ae-9cd8-34ae39115c0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:46.654081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60766d-14ca-45ae-9cd8-34ae39115c0e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:46.666135 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1193/2402] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:32:46.670515 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafc6ee-1b54-49e9-89d2-022010e55836 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:46.680973 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafc6ee-1b54-49e9-89d2-022010e55836 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:46.685548 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45316f9-89e9-4a4e-ad30-444979fd6da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:46.686142 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45316f9-89e9-4a4e-ad30-444979fd6da7 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:46.686421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45316f9-89e9-4a4e-ad30-444979fd6da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:46.686692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45316f9-89e9-4a4e-ad30-444979fd6da7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:46.687744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45316f9-89e9-4a4e-ad30-444979fd6da7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:46.689640 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45316f9-89e9-4a4e-ad30-444979fd6da7 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:46.716754 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1210/2403] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:46 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:46.721497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60766d-14ca-45ae-9cd8-34ae39115c0e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:46.731995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60766d-14ca-45ae-9cd8-34ae39115c0e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:46.784666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:46.790991 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550304f-78fb-43e4-8c52-675eb5f74e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:46.791653 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550304f-78fb-43e4-8c52-675eb5f74e3f admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:46.791939 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550304f-78fb-43e4-8c52-675eb5f74e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:46.792218 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550304f-78fb-43e4-8c52-675eb5f74e3f admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:46.792845 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550304f-78fb-43e4-8c52-675eb5f74e3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:46.794763 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550304f-78fb-43e4-8c52-675eb5f74e3f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:46.815926 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1194/2404] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:46 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:46.820151 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45316f9-89e9-4a4e-ad30-444979fd6da7 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:46.831367 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45316f9-89e9-4a4e-ad30-444979fd6da7 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:46.836006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70559ea5-c7aa-4d88-ae01-670214f23eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:46.836604 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70559ea5-c7aa-4d88-ae01-670214f23eed admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:46.836881 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70559ea5-c7aa-4d88-ae01-670214f23eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:46.837153 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70559ea5-c7aa-4d88-ae01-670214f23eed admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:46.837788 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70559ea5-c7aa-4d88-ae01-670214f23eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:46.839417 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70559ea5-c7aa-4d88-ae01-670214f23eed admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:46.850491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1211/2405] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:46.854406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550304f-78fb-43e4-8c52-675eb5f74e3f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:46.864810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550304f-78fb-43e4-8c52-675eb5f74e3f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:46.869494 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bba10-7fa1-499d-9e0b-6b2fa39cf01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:46.870092 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bba10-7fa1-499d-9e0b-6b2fa39cf01e admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:46.870383 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bba10-7fa1-499d-9e0b-6b2fa39cf01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:46.870680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bba10-7fa1-499d-9e0b-6b2fa39cf01e admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:46.871382 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bba10-7fa1-499d-9e0b-6b2fa39cf01e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:46.872975 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bba10-7fa1-499d-9e0b-6b2fa39cf01e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:46.884875 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1195/2406] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:46.889069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70559ea5-c7aa-4d88-ae01-670214f23eed admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:46.899529 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70559ea5-c7aa-4d88-ae01-670214f23eed admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:46.904264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9b04a-484a-4ddb-9962-1bb7a6cdecc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:46.904942 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9b04a-484a-4ddb-9962-1bb7a6cdecc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:46.905222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9b04a-484a-4ddb-9962-1bb7a6cdecc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:46.905498 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9b04a-484a-4ddb-9962-1bb7a6cdecc9 admin admin] PATH_INFO: `/v3/projects/54f032802a824478a7629e48695c8139/users/42cf528bdb0e4e54ac0a1f8130d66713/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:46.908683 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:46.923289 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9b04a-484a-4ddb-9962-1bb7a6cdecc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f032802a824478a7629e48695c8139, user_id=42cf528bdb0e4e54ac0a1f8130d66713, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:46.926039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9b04a-484a-4ddb-9962-1bb7a6cdecc9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:46.938853 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1212/2407] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:46 2022] PUT /identity/v3/projects/54f032802a824478a7629e48695c8139/users/42cf528bdb0e4e54ac0a1f8130d66713/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:46.950829 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35a3a9-a44f-4401-a04a-209fee231e8c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:46.951244 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35a3a9-a44f-4401-a04a-209fee231e8c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:46.951517 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35a3a9-a44f-4401-a04a-209fee231e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:46.981769 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-de35a3a9-a44f-4401-a04a-209fee231e8c None None] MFA Rules not processed for user `42cf528bdb0e4e54ac0a1f8130d66713`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:46.982991 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-de35a3a9-a44f-4401-a04a-209fee231e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:47.048363 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1196/2408] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:47.055091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9b04a-484a-4ddb-9962-1bb7a6cdecc9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:47.056800 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-05b9b04a-484a-4ddb-9962-1bb7a6cdecc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:47.059786 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35a3a9-a44f-4401-a04a-209fee231e8c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:47.061099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-de35a3a9-a44f-4401-a04a-209fee231e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:47.120405 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35a3a9-a44f-4401-a04a-209fee231e8c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:47.121700 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9b04a-484a-4ddb-9962-1bb7a6cdecc9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:47.126476 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e656c2f-893d-4bc0-9646-4ebfe9a85da5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:47.127006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526361b-48c8-44b3-b7c6-adad2b7c0a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:47.127167 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e656c2f-893d-4bc0-9646-4ebfe9a85da5 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:47.127428 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e656c2f-893d-4bc0-9646-4ebfe9a85da5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:47.127662 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526361b-48c8-44b3-b7c6-adad2b7c0a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:47.127772 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e656c2f-893d-4bc0-9646-4ebfe9a85da5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:47.127938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526361b-48c8-44b3-b7c6-adad2b7c0a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:47.128262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526361b-48c8-44b3-b7c6-adad2b7c0a5e admin admin] PATH_INFO: `/v3/users/6bc13ca4133042b7bfb92a8710b782a5` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:47.129339 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526361b-48c8-44b3-b7c6-adad2b7c0a5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc13ca4133042b7bfb92a8710b782a5)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:47.131279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526361b-48c8-44b3-b7c6-adad2b7c0a5e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:47.136188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e656c2f-893d-4bc0-9646-4ebfe9a85da5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:47.137620 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e656c2f-893d-4bc0-9646-4ebfe9a85da5 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:47.147430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1197/2409] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:32:47.183983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1526361b-48c8-44b3-b7c6-adad2b7c0a5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc13ca4133042b7bfb92a8710b782a5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:47.193104 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1526361b-48c8-44b3-b7c6-adad2b7c0a5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc13ca4133042b7bfb92a8710b782a5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:47.208576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1526361b-48c8-44b3-b7c6-adad2b7c0a5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc13ca4133042b7bfb92a8710b782a5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:47.216114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1526361b-48c8-44b3-b7c6-adad2b7c0a5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc13ca4133042b7bfb92a8710b782a5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:47.216832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-1526361b-48c8-44b3-b7c6-adad2b7c0a5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc13ca4133042b7bfb92a8710b782a5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:47.226295 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1213/2410] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:47 2022] DELETE /identity/v3/users/6bc13ca4133042b7bfb92a8710b782a5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:47.228001 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc1237-379a-4088-a793-07101198f0b2 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:47.228475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc1237-379a-4088-a793-07101198f0b2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:47.228891 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc1237-379a-4088-a793-07101198f0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:47.257641 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5edc1237-379a-4088-a793-07101198f0b2 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:47.259305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5edc1237-379a-4088-a793-07101198f0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:47.337037 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1198/2411] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:47.343928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526361b-48c8-44b3-b7c6-adad2b7c0a5e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:47.345687 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1526361b-48c8-44b3-b7c6-adad2b7c0a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:47.416371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526361b-48c8-44b3-b7c6-adad2b7c0a5e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:47.421219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f549f6c-43db-4421-861e-2a1e9560a851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:47.423852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f549f6c-43db-4421-861e-2a1e9560a851 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:47.424268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f549f6c-43db-4421-861e-2a1e9560a851 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:47.424622 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f549f6c-43db-4421-861e-2a1e9560a851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:47.442417 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f549f6c-43db-4421-861e-2a1e9560a851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:47.444891 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f549f6c-43db-4421-861e-2a1e9560a851 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:47.457744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1214/2412] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:47.750686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc1237-379a-4088-a793-07101198f0b2 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:47.752243 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5edc1237-379a-4088-a793-07101198f0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:47.764466 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc1237-379a-4088-a793-07101198f0b2 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:47.770717 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:47.771415 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:47.772237 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:47.772536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] PATH_INFO: `/v3/projects/a089bd6924994721aabc5156ac4d83e3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:47.773917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a089bd6924994721aabc5156ac4d83e3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:47.776023 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:47.778805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:47.826087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a089bd6924994721aabc5156ac4d83e3', 'request_id': 'req-d260a72c-da54-4ba3-8526-c01df73a1c9d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:47.832217 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a089bd6924994721aabc5156ac4d83e3', 'request_id': 'req-d260a72c-da54-4ba3-8526-c01df73a1c9d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:47.838257 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a089bd6924994721aabc5156ac4d83e3', 'request_id': 'req-d260a72c-da54-4ba3-8526-c01df73a1c9d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:47.850597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] The token cache is being invalidate because project a089bd6924994721aabc5156ac4d83e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:47.850882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:47.851520 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:47.852992 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1199/2413] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:47 2022] DELETE /identity/v3/projects/a089bd6924994721aabc5156ac4d83e3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:47.857294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f549f6c-43db-4421-861e-2a1e9560a851 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:47.863923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:47.867928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f549f6c-43db-4421-861e-2a1e9560a851 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:47.924217 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba948a3-6ade-4a35-88e6-e11c4937f82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:47.924843 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba948a3-6ade-4a35-88e6-e11c4937f82a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:47.925118 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba948a3-6ade-4a35-88e6-e11c4937f82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:47.925387 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba948a3-6ade-4a35-88e6-e11c4937f82a admin admin] PATH_INFO: `/v3/users/8520b3a76f824c74ae440d44aa490c7d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:47.926472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba948a3-6ade-4a35-88e6-e11c4937f82a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8520b3a76f824c74ae440d44aa490c7d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:47.928407 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba948a3-6ade-4a35-88e6-e11c4937f82a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:47.973651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bba948a3-6ade-4a35-88e6-e11c4937f82a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8520b3a76f824c74ae440d44aa490c7d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:47.981033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bba948a3-6ade-4a35-88e6-e11c4937f82a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8520b3a76f824c74ae440d44aa490c7d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:47.992783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bba948a3-6ade-4a35-88e6-e11c4937f82a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8520b3a76f824c74ae440d44aa490c7d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:48.000793 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bba948a3-6ade-4a35-88e6-e11c4937f82a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8520b3a76f824c74ae440d44aa490c7d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:48.001452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bba948a3-6ade-4a35-88e6-e11c4937f82a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8520b3a76f824c74ae440d44aa490c7d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:48.007512 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1215/2414] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:47 2022] DELETE /identity/v3/users/8520b3a76f824c74ae440d44aa490c7d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:48.271397 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:48.272814 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:48.286335 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260a72c-da54-4ba3-8526-c01df73a1c9d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:48.343969 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:48.345111 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:48.345401 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:48.345680 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe admin admin] PATH_INFO: `/v3/projects/4fed12f373184ae59c2bead68c781d26` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:48.346846 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fed12f373184ae59c2bead68c781d26)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:48.348801 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:48.395953 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fed12f373184ae59c2bead68c781d26', 'request_id': 'req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:48.401350 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fed12f373184ae59c2bead68c781d26', 'request_id': 'req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:48.407871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fed12f373184ae59c2bead68c781d26', 'request_id': 'req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:48.419641 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe admin admin] The token cache is being invalidate because project 4fed12f373184ae59c2bead68c781d26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:48.419929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:48.420543 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af6965b4-acfc-4edf-b7fa-6cfa5c9ff6fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:48.422991 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1200/2415] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:48 2022] DELETE /identity/v3/projects/4fed12f373184ae59c2bead68c781d26 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:48.429022 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feeda6a-8d50-4ca9-8629-5c1ab9314db5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:48.429338 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feeda6a-8d50-4ca9-8629-5c1ab9314db5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:48.429611 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feeda6a-8d50-4ca9-8629-5c1ab9314db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:48.449601 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2feeda6a-8d50-4ca9-8629-5c1ab9314db5 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:48.450798 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2feeda6a-8d50-4ca9-8629-5c1ab9314db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:48.508719 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1216/2416] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:48.518217 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9a483-3e65-4aaf-bdb0-b82a72a44d71 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:48.518608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9a483-3e65-4aaf-bdb0-b82a72a44d71 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:48.518791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9a483-3e65-4aaf-bdb0-b82a72a44d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:48.538925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-6bf9a483-3e65-4aaf-bdb0-b82a72a44d71 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:48.540159 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6bf9a483-3e65-4aaf-bdb0-b82a72a44d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:48.557058 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1201/2417] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:48.572316 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feeda6a-8d50-4ca9-8629-5c1ab9314db5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:48.583354 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feeda6a-8d50-4ca9-8629-5c1ab9314db5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:48.587923 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1735f0a-ad50-45d6-9ab5-71ebe2b4bbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:48.588748 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1735f0a-ad50-45d6-9ab5-71ebe2b4bbab admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:48.589020 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1735f0a-ad50-45d6-9ab5-71ebe2b4bbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:48.589469 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1735f0a-ad50-45d6-9ab5-71ebe2b4bbab admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:48.590148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1735f0a-ad50-45d6-9ab5-71ebe2b4bbab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:32:48.590446 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1735f0a-ad50-45d6-9ab5-71ebe2b4bbab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:48.592119 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1735f0a-ad50-45d6-9ab5-71ebe2b4bbab admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:48.604500 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1217/2418] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:32:48.609863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9a483-3e65-4aaf-bdb0-b82a72a44d71 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:48.621521 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9a483-3e65-4aaf-bdb0-b82a72a44d71 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:48.628092 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdb12d-973c-4a99-9fac-9bc052b9e187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:48.628839 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdb12d-973c-4a99-9fac-9bc052b9e187 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:48.629191 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdb12d-973c-4a99-9fac-9bc052b9e187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:48.629589 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdb12d-973c-4a99-9fac-9bc052b9e187 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:48.630299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fdb12d-973c-4a99-9fac-9bc052b9e187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:48.632346 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fdb12d-973c-4a99-9fac-9bc052b9e187 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:48.652362 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:48.678087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1202/2419] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:48 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:32:48.679030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1735f0a-ad50-45d6-9ab5-71ebe2b4bbab admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:48.689797 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1735f0a-ad50-45d6-9ab5-71ebe2b4bbab admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:48.699167 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:48.759502 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048b444-09a2-48af-9faa-ce86e5f7ab62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:48.760166 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048b444-09a2-48af-9faa-ce86e5f7ab62 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:48.760388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048b444-09a2-48af-9faa-ce86e5f7ab62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:48.760661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048b444-09a2-48af-9faa-ce86e5f7ab62 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:48.761300 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048b444-09a2-48af-9faa-ce86e5f7ab62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:48.763314 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048b444-09a2-48af-9faa-ce86e5f7ab62 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:48.787150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1218/2420] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:48 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:32:48.791439 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdb12d-973c-4a99-9fac-9bc052b9e187 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:48.803939 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdb12d-973c-4a99-9fac-9bc052b9e187 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:48.809068 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aed79-c792-40af-9ae0-2bb7afd32ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:48.809682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aed79-c792-40af-9ae0-2bb7afd32ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:48.809987 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aed79-c792-40af-9ae0-2bb7afd32ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:48.810225 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aed79-c792-40af-9ae0-2bb7afd32ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:48.810867 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aed79-c792-40af-9ae0-2bb7afd32ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:48.812567 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aed79-c792-40af-9ae0-2bb7afd32ba1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:48.828570 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1203/2421] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:48.832518 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048b444-09a2-48af-9faa-ce86e5f7ab62 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:48.843245 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048b444-09a2-48af-9faa-ce86e5f7ab62 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:48.849441 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0be06-6306-4a72-a647-20c4c7044bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:48.850059 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0be06-6306-4a72-a647-20c4c7044bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:48.850325 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0be06-6306-4a72-a647-20c4c7044bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:48.850626 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0be06-6306-4a72-a647-20c4c7044bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:48.851386 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0be06-6306-4a72-a647-20c4c7044bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:48.853708 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0be06-6306-4a72-a647-20c4c7044bb5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:48.867402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1219/2422] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:48.871168 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aed79-c792-40af-9ae0-2bb7afd32ba1 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:48.885876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aed79-c792-40af-9ae0-2bb7afd32ba1 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:48.892497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c078f36-6774-4607-acda-3f0d4421b225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:48.893180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c078f36-6774-4607-acda-3f0d4421b225 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:48.893491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c078f36-6774-4607-acda-3f0d4421b225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:48.893799 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c078f36-6774-4607-acda-3f0d4421b225 admin admin] PATH_INFO: `/v3/projects/7940e618534540d78e2801fe55d0f824/users/7b1685286ae94b80941b1b1fcfa2d7fa/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:48.917078 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c078f36-6774-4607-acda-3f0d4421b225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7940e618534540d78e2801fe55d0f824, user_id=7b1685286ae94b80941b1b1fcfa2d7fa, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:48.920115 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c078f36-6774-4607-acda-3f0d4421b225 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:48.936125 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1204/2423] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:48 2022] PUT /identity/v3/projects/7940e618534540d78e2801fe55d0f824/users/7b1685286ae94b80941b1b1fcfa2d7fa/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:48.948076 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d91ad-5efa-4534-bac8-009a3bae3b30 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:48.948366 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d91ad-5efa-4534-bac8-009a3bae3b30 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:48.948701 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d91ad-5efa-4534-bac8-009a3bae3b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:48.986159 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-335d91ad-5efa-4534-bac8-009a3bae3b30 None None] MFA Rules not processed for user `7b1685286ae94b80941b1b1fcfa2d7fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:48.987471 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-335d91ad-5efa-4534-bac8-009a3bae3b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:49.060355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1220/2424] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:49.073163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c078f36-6774-4607-acda-3f0d4421b225 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:49.074849 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9c078f36-6774-4607-acda-3f0d4421b225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:49.134048 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c078f36-6774-4607-acda-3f0d4421b225 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:49.137856 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3182c-576d-453a-a4e2-7329ab7fbc89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:49.138501 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3182c-576d-453a-a4e2-7329ab7fbc89 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:49.138786 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3182c-576d-453a-a4e2-7329ab7fbc89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:49.139058 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3182c-576d-453a-a4e2-7329ab7fbc89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:49.148761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3182c-576d-453a-a4e2-7329ab7fbc89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:49.150192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3182c-576d-453a-a4e2-7329ab7fbc89 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:49.158999 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1205/2425] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:49.180858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d91ad-5efa-4534-bac8-009a3bae3b30 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:49.245407 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d91ad-5efa-4534-bac8-009a3bae3b30 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:49.250616 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3e5de-8ec7-4ec3-99bf-18b90d3786fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:49.251452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3e5de-8ec7-4ec3-99bf-18b90d3786fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:49.251784 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3e5de-8ec7-4ec3-99bf-18b90d3786fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:49.252063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3e5de-8ec7-4ec3-99bf-18b90d3786fc admin admin] PATH_INFO: `/v3/users/7b1685286ae94b80941b1b1fcfa2d7fa` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:49.253302 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3e5de-8ec7-4ec3-99bf-18b90d3786fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b1685286ae94b80941b1b1fcfa2d7fa)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:49.255295 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3e5de-8ec7-4ec3-99bf-18b90d3786fc admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:49.277310 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:49.304195 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b6f3e5de-8ec7-4ec3-99bf-18b90d3786fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b1685286ae94b80941b1b1fcfa2d7fa'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:49.311101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b6f3e5de-8ec7-4ec3-99bf-18b90d3786fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b1685286ae94b80941b1b1fcfa2d7fa'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:49.324034 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b6f3e5de-8ec7-4ec3-99bf-18b90d3786fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b1685286ae94b80941b1b1fcfa2d7fa'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:49.331253 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b6f3e5de-8ec7-4ec3-99bf-18b90d3786fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b1685286ae94b80941b1b1fcfa2d7fa'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:49.331929 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b6f3e5de-8ec7-4ec3-99bf-18b90d3786fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b1685286ae94b80941b1b1fcfa2d7fa'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:49.338232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1221/2426] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:49 2022] DELETE /identity/v3/users/7b1685286ae94b80941b1b1fcfa2d7fa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:49.342687 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38fbe6-562b-4f5a-a550-bfb37f4e7b0b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:49.343140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38fbe6-562b-4f5a-a550-bfb37f4e7b0b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:49.343780 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38fbe6-562b-4f5a-a550-bfb37f4e7b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:49.363272 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-cf38fbe6-562b-4f5a-a550-bfb37f4e7b0b None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:49.364461 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cf38fbe6-562b-4f5a-a550-bfb37f4e7b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:49.423093 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1206/2427] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:49.432616 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3e5de-8ec7-4ec3-99bf-18b90d3786fc admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:49.434016 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b6f3e5de-8ec7-4ec3-99bf-18b90d3786fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:49.492290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3e5de-8ec7-4ec3-99bf-18b90d3786fc admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:49.495920 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d18c85-09b4-459a-92aa-ad57385c5da4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:49.496522 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d18c85-09b4-459a-92aa-ad57385c5da4 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:49.496799 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d18c85-09b4-459a-92aa-ad57385c5da4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:49.497072 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d18c85-09b4-459a-92aa-ad57385c5da4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:49.509996 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d18c85-09b4-459a-92aa-ad57385c5da4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:49.511525 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d18c85-09b4-459a-92aa-ad57385c5da4 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:49.521099 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1222/2428] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:49.788663 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38fbe6-562b-4f5a-a550-bfb37f4e7b0b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:49.790100 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cf38fbe6-562b-4f5a-a550-bfb37f4e7b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:49.802473 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38fbe6-562b-4f5a-a550-bfb37f4e7b0b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:49.807142 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360c345-fae2-447a-b421-188d02ed4f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:49.807791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360c345-fae2-447a-b421-188d02ed4f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:49.808074 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360c345-fae2-447a-b421-188d02ed4f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:49.808347 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360c345-fae2-447a-b421-188d02ed4f3c admin admin] PATH_INFO: `/v3/projects/7940e618534540d78e2801fe55d0f824` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:49.809493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360c345-fae2-447a-b421-188d02ed4f3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7940e618534540d78e2801fe55d0f824)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:49.811504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360c345-fae2-447a-b421-188d02ed4f3c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:49.861203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0360c345-fae2-447a-b421-188d02ed4f3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7940e618534540d78e2801fe55d0f824', 'request_id': 'req-0360c345-fae2-447a-b421-188d02ed4f3c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:49.865850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:49.867318 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0360c345-fae2-447a-b421-188d02ed4f3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7940e618534540d78e2801fe55d0f824', 'request_id': 'req-0360c345-fae2-447a-b421-188d02ed4f3c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:49.875452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0360c345-fae2-447a-b421-188d02ed4f3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7940e618534540d78e2801fe55d0f824', 'request_id': 'req-0360c345-fae2-447a-b421-188d02ed4f3c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:49.887674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0360c345-fae2-447a-b421-188d02ed4f3c admin admin] The token cache is being invalidate because project 7940e618534540d78e2801fe55d0f824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:49.888145 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0360c345-fae2-447a-b421-188d02ed4f3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:49.888767 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0360c345-fae2-447a-b421-188d02ed4f3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:49.890938 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1207/2429] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:49 2022] DELETE /identity/v3/projects/7940e618534540d78e2801fe55d0f824 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:49.896058 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73d269-3c84-4ba9-8772-34ed7f2f604d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:49.896540 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73d269-3c84-4ba9-8772-34ed7f2f604d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:49.896633 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73d269-3c84-4ba9-8772-34ed7f2f604d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:49.917468 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1e73d269-3c84-4ba9-8772-34ed7f2f604d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:49.918654 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1e73d269-3c84-4ba9-8772-34ed7f2f604d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:49.976657 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1223/2430] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:49.984636 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23c23d-0f0a-4621-8b07-56934222a7eb None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:49.984925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23c23d-0f0a-4621-8b07-56934222a7eb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:49.985271 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23c23d-0f0a-4621-8b07-56934222a7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:50.004927 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7a23c23d-0f0a-4621-8b07-56934222a7eb None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:50.006145 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7a23c23d-0f0a-4621-8b07-56934222a7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:50.014180 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1208/2431] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:50.019303 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73d269-3c84-4ba9-8772-34ed7f2f604d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:50.032026 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73d269-3c84-4ba9-8772-34ed7f2f604d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:50.040096 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59470e-aa83-4306-b1c5-988ad3b1dce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:50.040642 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59470e-aa83-4306-b1c5-988ad3b1dce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:50.040915 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59470e-aa83-4306-b1c5-988ad3b1dce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:50.041283 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59470e-aa83-4306-b1c5-988ad3b1dce3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:50.041855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59470e-aa83-4306-b1c5-988ad3b1dce3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:32:50.042155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59470e-aa83-4306-b1c5-988ad3b1dce3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:50.043893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59470e-aa83-4306-b1c5-988ad3b1dce3 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:50.052699 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23c23d-0f0a-4621-8b07-56934222a7eb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:50.054278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7a23c23d-0f0a-4621-8b07-56934222a7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:50.064344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1224/2432] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:32:50.065168 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59470e-aa83-4306-b1c5-988ad3b1dce3 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:50.068430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23c23d-0f0a-4621-8b07-56934222a7eb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:50.073955 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbbe4c-ee96-4e27-abbc-67a9e85ee2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:50.074694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbbe4c-ee96-4e27-abbc-67a9e85ee2eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:50.074918 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbbe4c-ee96-4e27-abbc-67a9e85ee2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:50.075250 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbbe4c-ee96-4e27-abbc-67a9e85ee2eb admin admin] PATH_INFO: `/v3/users/bd8bba63a2f04fa78a4794fe7fc1079b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:50.075357 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59470e-aa83-4306-b1c5-988ad3b1dce3 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:50.076446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcbbe4c-ee96-4e27-abbc-67a9e85ee2eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd8bba63a2f04fa78a4794fe7fc1079b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:50.079081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcbbe4c-ee96-4e27-abbc-67a9e85ee2eb admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:50.081213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebad919-620b-406e-8438-296a90e71ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:50.081850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebad919-620b-406e-8438-296a90e71ffa admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:50.082096 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebad919-620b-406e-8438-296a90e71ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:50.082364 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebad919-620b-406e-8438-296a90e71ffa admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:50.082985 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebad919-620b-406e-8438-296a90e71ffa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:50.085000 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebad919-620b-406e-8438-296a90e71ffa admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:50.123238 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1225/2433] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:50 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:50.126524 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebad919-620b-406e-8438-296a90e71ffa admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:50.134659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fbcbbe4c-ee96-4e27-abbc-67a9e85ee2eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd8bba63a2f04fa78a4794fe7fc1079b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:50.136047 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:50.144472 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fbcbbe4c-ee96-4e27-abbc-67a9e85ee2eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd8bba63a2f04fa78a4794fe7fc1079b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:50.145449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fbcbbe4c-ee96-4e27-abbc-67a9e85ee2eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd8bba63a2f04fa78a4794fe7fc1079b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:50.159554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fbcbbe4c-ee96-4e27-abbc-67a9e85ee2eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd8bba63a2f04fa78a4794fe7fc1079b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:50.171299 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fbcbbe4c-ee96-4e27-abbc-67a9e85ee2eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd8bba63a2f04fa78a4794fe7fc1079b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:50.182266 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1209/2434] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:50 2022] DELETE /identity/v3/users/bd8bba63a2f04fa78a4794fe7fc1079b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:50.186923 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adb674-6e7a-4020-b5f2-371d550b6213 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:50.187725 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adb674-6e7a-4020-b5f2-371d550b6213 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:50.188038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adb674-6e7a-4020-b5f2-371d550b6213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:50.198911 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebad919-620b-406e-8438-296a90e71ffa admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:50.209609 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-28adb674-6e7a-4020-b5f2-371d550b6213 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:50.210933 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-28adb674-6e7a-4020-b5f2-371d550b6213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:50.256852 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d35c-bd68-473e-9025-eb69019fca0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:50.257557 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d35c-bd68-473e-9025-eb69019fca0c admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:50.257956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d35c-bd68-473e-9025-eb69019fca0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:50.258447 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d35c-bd68-473e-9025-eb69019fca0c admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:50.259093 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689d35c-bd68-473e-9025-eb69019fca0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:50.261082 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689d35c-bd68-473e-9025-eb69019fca0c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:50.280297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1210/2435] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:50.287219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1226/2436] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:50 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:50.291225 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d35c-bd68-473e-9025-eb69019fca0c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:50.291502 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adb674-6e7a-4020-b5f2-371d550b6213 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:50.292968 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3689d35c-bd68-473e-9025-eb69019fca0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:50.293419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-28adb674-6e7a-4020-b5f2-371d550b6213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:50.305932 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adb674-6e7a-4020-b5f2-371d550b6213 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:50.310685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98674947-870e-4d5f-8a6c-be9df11555dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:50.311326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98674947-870e-4d5f-8a6c-be9df11555dc admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:50.311635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98674947-870e-4d5f-8a6c-be9df11555dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:50.311912 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98674947-870e-4d5f-8a6c-be9df11555dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:50.312641 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98674947-870e-4d5f-8a6c-be9df11555dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:50.314291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98674947-870e-4d5f-8a6c-be9df11555dc admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:50.327122 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1211/2437] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:50.331324 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98674947-870e-4d5f-8a6c-be9df11555dc admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:50.341942 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98674947-870e-4d5f-8a6c-be9df11555dc admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:50.348651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edef567-39c6-40fd-b83c-bd51925b538f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:50.349281 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edef567-39c6-40fd-b83c-bd51925b538f admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:50.349533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edef567-39c6-40fd-b83c-bd51925b538f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:50.349812 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edef567-39c6-40fd-b83c-bd51925b538f admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:50.350456 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edef567-39c6-40fd-b83c-bd51925b538f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:50.352140 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edef567-39c6-40fd-b83c-bd51925b538f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:50.363387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1212/2438] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:32:50.367901 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edef567-39c6-40fd-b83c-bd51925b538f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:50.368225 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d35c-bd68-473e-9025-eb69019fca0c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:50.372749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f35564-d3fb-4b75-a179-b5e1d8747d3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:50.373449 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f35564-d3fb-4b75-a179-b5e1d8747d3a service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:50.374055 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f35564-d3fb-4b75-a179-b5e1d8747d3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:50.374281 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f35564-d3fb-4b75-a179-b5e1d8747d3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:50.380256 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edef567-39c6-40fd-b83c-bd51925b538f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:50.386178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c34d20-a27c-425e-bae5-d1571a783619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:50.387534 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c34d20-a27c-425e-bae5-d1571a783619 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:50.387917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c34d20-a27c-425e-bae5-d1571a783619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:50.388056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f35564-d3fb-4b75-a179-b5e1d8747d3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:50.388475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c34d20-a27c-425e-bae5-d1571a783619 admin admin] PATH_INFO: `/v3/projects/304e0188d24a4b79846b2b63adc7b847/users/84d6928ba28642858d870450eded6ef0/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:50.389452 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f35564-d3fb-4b75-a179-b5e1d8747d3a service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:50.402262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1227/2439] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:50.405053 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c34d20-a27c-425e-bae5-d1571a783619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=304e0188d24a4b79846b2b63adc7b847, user_id=84d6928ba28642858d870450eded6ef0, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:50.409133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c34d20-a27c-425e-bae5-d1571a783619 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:50.439241 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1213/2440] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:50 2022] PUT /identity/v3/projects/304e0188d24a4b79846b2b63adc7b847/users/84d6928ba28642858d870450eded6ef0/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:32:50.449500 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c23f9-93a9-49d7-ba33-de9f5a796d15 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:50.449788 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c23f9-93a9-49d7-ba33-de9f5a796d15 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:50.450152 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c23f9-93a9-49d7-ba33-de9f5a796d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:50.485178 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fd5c23f9-93a9-49d7-ba33-de9f5a796d15 None None] MFA Rules not processed for user `84d6928ba28642858d870450eded6ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:50.486961 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fd5c23f9-93a9-49d7-ba33-de9f5a796d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:50.534332 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:50.571115 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1228/2441] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:50.585161 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c34d20-a27c-425e-bae5-d1571a783619 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:50.586917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-35c34d20-a27c-425e-bae5-d1571a783619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:50.593977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:50.649331 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c34d20-a27c-425e-bae5-d1571a783619 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:50.653031 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145c1ec-87f4-4585-9b67-89a1adcc47dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c05a1b303c2418fb2941b8ef56f8972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:50.653650 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145c1ec-87f4-4585-9b67-89a1adcc47dc service glance] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:50.653928 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145c1ec-87f4-4585-9b67-89a1adcc47dc service glance] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:50.654198 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145c1ec-87f4-4585-9b67-89a1adcc47dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:50.663101 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145c1ec-87f4-4585-9b67-89a1adcc47dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:50.664752 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145c1ec-87f4-4585-9b67-89a1adcc47dc service glance] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:50.675228 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1214/2442] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:50.708856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c23f9-93a9-49d7-ba33-de9f5a796d15 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:50.710739 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fd5c23f9-93a9-49d7-ba33-de9f5a796d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:50.789851 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c23f9-93a9-49d7-ba33-de9f5a796d15 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:50.794753 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:50.795436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:50.795743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:50.796018 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] PATH_INFO: `/v3/projects/336f450089ad4487b7d0646391f6fdf5` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:50.797928 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=336f450089ad4487b7d0646391f6fdf5)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:50.800017 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:50.850579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '336f450089ad4487b7d0646391f6fdf5', 'request_id': 'req-84237a42-3a66-4eff-ae5c-dc494e5062f3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:50.856625 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '336f450089ad4487b7d0646391f6fdf5', 'request_id': 'req-84237a42-3a66-4eff-ae5c-dc494e5062f3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:50.863952 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '336f450089ad4487b7d0646391f6fdf5', 'request_id': 'req-84237a42-3a66-4eff-ae5c-dc494e5062f3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:50.878165 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] The token cache is being invalidate because project 336f450089ad4487b7d0646391f6fdf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:50.878457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:50.879245 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:50.880690 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1229/2443] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:50 2022] DELETE /identity/v3/projects/336f450089ad4487b7d0646391f6fdf5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:50.884569 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145c1ec-87f4-4585-9b67-89a1adcc47dc service glance] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:50.895613 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145c1ec-87f4-4585-9b67-89a1adcc47dc service glance] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:50.955356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18504b-aea3-42a1-a6e1-942c45687b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:50.955922 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18504b-aea3-42a1-a6e1-942c45687b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:50.956200 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18504b-aea3-42a1-a6e1-942c45687b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:50.956468 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18504b-aea3-42a1-a6e1-942c45687b35 admin admin] PATH_INFO: `/v3/users/993a936a9c5c4fa7aaf7a8b151ade1ce` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:50.957602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18504b-aea3-42a1-a6e1-942c45687b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=993a936a9c5c4fa7aaf7a8b151ade1ce)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:50.959639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18504b-aea3-42a1-a6e1-942c45687b35 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:51.010171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fc18504b-aea3-42a1-a6e1-942c45687b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993a936a9c5c4fa7aaf7a8b151ade1ce'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.017575 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fc18504b-aea3-42a1-a6e1-942c45687b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993a936a9c5c4fa7aaf7a8b151ade1ce'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.018250 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fc18504b-aea3-42a1-a6e1-942c45687b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993a936a9c5c4fa7aaf7a8b151ade1ce'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.032269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fc18504b-aea3-42a1-a6e1-942c45687b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993a936a9c5c4fa7aaf7a8b151ade1ce'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.040009 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fc18504b-aea3-42a1-a6e1-942c45687b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993a936a9c5c4fa7aaf7a8b151ade1ce'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.047089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1215/2444] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:50 2022] DELETE /identity/v3/users/993a936a9c5c4fa7aaf7a8b151ade1ce => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:51.626264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:51.628056 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:51.641959 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84237a42-3a66-4eff-ae5c-dc494e5062f3 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:51.712561 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:51.713214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:51.713494 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:51.713782 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] PATH_INFO: `/v3/projects/7622ec08a8064bc6ba542b330de09b24` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:51.714945 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] RBAC: Authorizing `identity:delete_project(project_id=7622ec08a8064bc6ba542b330de09b24)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:51.716978 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:51.744155 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:51.777731 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7622ec08a8064bc6ba542b330de09b24', 'request_id': 'req-928a39f9-94d4-479c-aed3-407e1096f9db'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.784577 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7622ec08a8064bc6ba542b330de09b24', 'request_id': 'req-928a39f9-94d4-479c-aed3-407e1096f9db'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.794095 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7622ec08a8064bc6ba542b330de09b24', 'request_id': 'req-928a39f9-94d4-479c-aed3-407e1096f9db'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.808247 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] The token cache is being invalidate because project 7622ec08a8064bc6ba542b330de09b24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:51.808533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.809146 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.810609 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1230/2445] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:51 2022] DELETE /identity/v3/projects/7622ec08a8064bc6ba542b330de09b24 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:51.814726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18504b-aea3-42a1-a6e1-942c45687b35 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:51.816291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-fc18504b-aea3-42a1-a6e1-942c45687b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:51.829308 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18504b-aea3-42a1-a6e1-942c45687b35 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:51.856382 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:51.894014 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d726642-486f-4374-9d5f-ad8419599879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:51.894682 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d726642-486f-4374-9d5f-ad8419599879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:51.894963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d726642-486f-4374-9d5f-ad8419599879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:51.895263 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d726642-486f-4374-9d5f-ad8419599879 admin admin] PATH_INFO: `/v3/users/60e9408fa6114130bc33255c9d94e625` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:51.896479 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d726642-486f-4374-9d5f-ad8419599879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60e9408fa6114130bc33255c9d94e625)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:51.898283 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d726642-486f-4374-9d5f-ad8419599879 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:51.951429 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7d726642-486f-4374-9d5f-ad8419599879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60e9408fa6114130bc33255c9d94e625'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.959452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7d726642-486f-4374-9d5f-ad8419599879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60e9408fa6114130bc33255c9d94e625'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.960139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7d726642-486f-4374-9d5f-ad8419599879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60e9408fa6114130bc33255c9d94e625'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.972662 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7d726642-486f-4374-9d5f-ad8419599879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60e9408fa6114130bc33255c9d94e625'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.985001 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7d726642-486f-4374-9d5f-ad8419599879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60e9408fa6114130bc33255c9d94e625'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:51.992154 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1216/2446] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:51 2022] DELETE /identity/v3/users/60e9408fa6114130bc33255c9d94e625 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:52.308011 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:52.309613 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:52.326671 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a39f9-94d4-479c-aed3-407e1096f9db admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:52.395260 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fabe5-c58e-4923-a741-7eac07f3077f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:52.395857 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fabe5-c58e-4923-a741-7eac07f3077f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:52.396175 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fabe5-c58e-4923-a741-7eac07f3077f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:52.396427 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fabe5-c58e-4923-a741-7eac07f3077f admin admin] PATH_INFO: `/v3/projects/4a01e1dab6644e1797112dad1df6480f` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:52.397543 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fabe5-c58e-4923-a741-7eac07f3077f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a01e1dab6644e1797112dad1df6480f)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:52.399484 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fabe5-c58e-4923-a741-7eac07f3077f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:52.451408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-928fabe5-c58e-4923-a741-7eac07f3077f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a01e1dab6644e1797112dad1df6480f', 'request_id': 'req-928fabe5-c58e-4923-a741-7eac07f3077f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:52.459184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-928fabe5-c58e-4923-a741-7eac07f3077f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a01e1dab6644e1797112dad1df6480f', 'request_id': 'req-928fabe5-c58e-4923-a741-7eac07f3077f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:52.465809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-928fabe5-c58e-4923-a741-7eac07f3077f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a01e1dab6644e1797112dad1df6480f', 'request_id': 'req-928fabe5-c58e-4923-a741-7eac07f3077f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:52.481809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-928fabe5-c58e-4923-a741-7eac07f3077f admin admin] The token cache is being invalidate because project 4a01e1dab6644e1797112dad1df6480f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:52.482130 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-928fabe5-c58e-4923-a741-7eac07f3077f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:52.482850 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-928fabe5-c58e-4923-a741-7eac07f3077f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:52.485496 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1231/2447] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:52 2022] DELETE /identity/v3/projects/4a01e1dab6644e1797112dad1df6480f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:55.934914 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d726642-486f-4374-9d5f-ad8419599879 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:55.936340 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7d726642-486f-4374-9d5f-ad8419599879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:55.949275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d726642-486f-4374-9d5f-ad8419599879 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:56.007520 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c0154-51d2-4d95-a3cf-2422e2949d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:56.008253 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c0154-51d2-4d95-a3cf-2422e2949d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:56.008533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c0154-51d2-4d95-a3cf-2422e2949d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:56.008805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c0154-51d2-4d95-a3cf-2422e2949d4e admin admin] PATH_INFO: `/v3/users/42cf528bdb0e4e54ac0a1f8130d66713` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:56.010005 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c0154-51d2-4d95-a3cf-2422e2949d4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=42cf528bdb0e4e54ac0a1f8130d66713)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:56.011963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c0154-51d2-4d95-a3cf-2422e2949d4e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:56.059777 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e15c0154-51d2-4d95-a3cf-2422e2949d4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42cf528bdb0e4e54ac0a1f8130d66713'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:56.061377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:56.069405 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e15c0154-51d2-4d95-a3cf-2422e2949d4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42cf528bdb0e4e54ac0a1f8130d66713'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:56.070085 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e15c0154-51d2-4d95-a3cf-2422e2949d4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42cf528bdb0e4e54ac0a1f8130d66713'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:56.084102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e15c0154-51d2-4d95-a3cf-2422e2949d4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42cf528bdb0e4e54ac0a1f8130d66713'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:56.090860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e15c0154-51d2-4d95-a3cf-2422e2949d4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42cf528bdb0e4e54ac0a1f8130d66713'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:56.096828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1217/2448] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:32:55 2022] DELETE /identity/v3/users/42cf528bdb0e4e54ac0a1f8130d66713 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:56.101246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e7eae-a06c-40b4-92e8-1ef8ed70341a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:56.101721 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e7eae-a06c-40b4-92e8-1ef8ed70341a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:56.101840 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e7eae-a06c-40b4-92e8-1ef8ed70341a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:56.121118 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3a8e7eae-a06c-40b4-92e8-1ef8ed70341a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:56.122359 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3a8e7eae-a06c-40b4-92e8-1ef8ed70341a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:56.144585 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:56.185394 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1232/2449] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:56.195782 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c0154-51d2-4d95-a3cf-2422e2949d4e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:56.197209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e15c0154-51d2-4d95-a3cf-2422e2949d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:56.255605 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c0154-51d2-4d95-a3cf-2422e2949d4e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:56.259320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f64081-2e47-4c4e-b021-f16274433ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:56.259874 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f64081-2e47-4c4e-b021-f16274433ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:56.260154 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f64081-2e47-4c4e-b021-f16274433ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:56.260430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f64081-2e47-4c4e-b021-f16274433ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:56.269838 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f64081-2e47-4c4e-b021-f16274433ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:56.271298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f64081-2e47-4c4e-b021-f16274433ae7 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:56.280480 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1218/2450] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:56.566290 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e7eae-a06c-40b4-92e8-1ef8ed70341a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:56.567907 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3a8e7eae-a06c-40b4-92e8-1ef8ed70341a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:56.581169 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e7eae-a06c-40b4-92e8-1ef8ed70341a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:56.585747 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:56.586372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:56.586639 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:56.586905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7 admin admin] PATH_INFO: `/v3/projects/54f032802a824478a7629e48695c8139` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:56.588036 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54f032802a824478a7629e48695c8139)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:56.589922 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:56.637077 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54f032802a824478a7629e48695c8139', 'request_id': 'req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:56.643956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54f032802a824478a7629e48695c8139', 'request_id': 'req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:56.650320 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54f032802a824478a7629e48695c8139', 'request_id': 'req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:56.665572 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7 admin admin] The token cache is being invalidate because project 54f032802a824478a7629e48695c8139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:32:56.665937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:56.666563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-543a5e2d-8c1c-403a-a5a0-b0b6e4b244a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:32:56.668122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1233/2451] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:32:56 2022] DELETE /identity/v3/projects/54f032802a824478a7629e48695c8139 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:56.673382 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14a903-ba77-49e0-8ab6-6df63c6808b0 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:56.673669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14a903-ba77-49e0-8ab6-6df63c6808b0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:56.673948 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14a903-ba77-49e0-8ab6-6df63c6808b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:56.693488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7f14a903-ba77-49e0-8ab6-6df63c6808b0 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:56.694685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7f14a903-ba77-49e0-8ab6-6df63c6808b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:56.757203 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1219/2452] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:56.767279 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffacd9-90f1-47e8-855f-2424fe5c9d3e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:56.767596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffacd9-90f1-47e8-855f-2424fe5c9d3e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:56.767888 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffacd9-90f1-47e8-855f-2424fe5c9d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:56.788618 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-64ffacd9-90f1-47e8-855f-2424fe5c9d3e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:56.789809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-64ffacd9-90f1-47e8-855f-2424fe5c9d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:56.797937 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1234/2453] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:56.802139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14a903-ba77-49e0-8ab6-6df63c6808b0 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:56.812377 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14a903-ba77-49e0-8ab6-6df63c6808b0 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:56.817562 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d4e9e-9bee-4cc2-975e-bccf4e34e442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:56.818168 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d4e9e-9bee-4cc2-975e-bccf4e34e442 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:56.818449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d4e9e-9bee-4cc2-975e-bccf4e34e442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:56.818723 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d4e9e-9bee-4cc2-975e-bccf4e34e442 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:56.819401 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d4e9e-9bee-4cc2-975e-bccf4e34e442 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:32:56.819710 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d4e9e-9bee-4cc2-975e-bccf4e34e442 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:56.821295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d4e9e-9bee-4cc2-975e-bccf4e34e442 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:56.834863 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1220/2454] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:32:56.839221 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffacd9-90f1-47e8-855f-2424fe5c9d3e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:56.848779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffacd9-90f1-47e8-855f-2424fe5c9d3e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:56.853347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e6260-59ea-487c-bf5e-27158c83dd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:56.853938 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e6260-59ea-487c-bf5e-27158c83dd08 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:56.854212 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e6260-59ea-487c-bf5e-27158c83dd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:56.854480 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e6260-59ea-487c-bf5e-27158c83dd08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:56.855127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e6260-59ea-487c-bf5e-27158c83dd08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:56.857297 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e6260-59ea-487c-bf5e-27158c83dd08 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:56.885378 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1235/2455] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:56 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:56.889704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d4e9e-9bee-4cc2-975e-bccf4e34e442 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:56.900415 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d4e9e-9bee-4cc2-975e-bccf4e34e442 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:56.950742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:56.959202 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4d8e5-b70c-481d-b594-f39f802c4e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:56.960222 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4d8e5-b70c-481d-b594-f39f802c4e0d admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:56.960564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4d8e5-b70c-481d-b594-f39f802c4e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:56.960917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4d8e5-b70c-481d-b594-f39f802c4e0d admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:56.961541 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4d8e5-b70c-481d-b594-f39f802c4e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:56.963822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4d8e5-b70c-481d-b594-f39f802c4e0d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:56.984410 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1221/2456] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:56 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:56.988722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e6260-59ea-487c-bf5e-27158c83dd08 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:56.999637 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e6260-59ea-487c-bf5e-27158c83dd08 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:57.004858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24faa31-8e70-43a2-ac4a-b57efc935778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:57.005465 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24faa31-8e70-43a2-ac4a-b57efc935778 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:57.005742 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24faa31-8e70-43a2-ac4a-b57efc935778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:57.006010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24faa31-8e70-43a2-ac4a-b57efc935778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:57.006645 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24faa31-8e70-43a2-ac4a-b57efc935778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:57.008339 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24faa31-8e70-43a2-ac4a-b57efc935778 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:57.018849 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1236/2457] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:57.022906 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4d8e5-b70c-481d-b594-f39f802c4e0d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:57.033280 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4d8e5-b70c-481d-b594-f39f802c4e0d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:57.037901 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0cb50-d227-4c37-9510-7051cf80f633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:57.038497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0cb50-d227-4c37-9510-7051cf80f633 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:57.038776 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0cb50-d227-4c37-9510-7051cf80f633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:57.039046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0cb50-d227-4c37-9510-7051cf80f633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:57.039712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0cb50-d227-4c37-9510-7051cf80f633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:57.041292 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0cb50-d227-4c37-9510-7051cf80f633 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:57.052953 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1222/2458] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:57.057174 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24faa31-8e70-43a2-ac4a-b57efc935778 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:57.068421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24faa31-8e70-43a2-ac4a-b57efc935778 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:57.073293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4910fc-2c73-4336-9a88-35f25ab9d43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:57.073941 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4910fc-2c73-4336-9a88-35f25ab9d43b admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:57.074216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4910fc-2c73-4336-9a88-35f25ab9d43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:57.074495 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4910fc-2c73-4336-9a88-35f25ab9d43b admin admin] PATH_INFO: `/v3/projects/097f22ab9d854cb39e0a75f632ae1362/users/e989ae02a7d74bb996b24bed89e7459a/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:57.089899 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4910fc-2c73-4336-9a88-35f25ab9d43b admin admin] RBAC: Authorizing `identity:create_grant(project_id=097f22ab9d854cb39e0a75f632ae1362, user_id=e989ae02a7d74bb996b24bed89e7459a, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:57.092676 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4910fc-2c73-4336-9a88-35f25ab9d43b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:57.107140 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1237/2459] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:57 2022] PUT /identity/v3/projects/097f22ab9d854cb39e0a75f632ae1362/users/e989ae02a7d74bb996b24bed89e7459a/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:57.112425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70dc6d-b032-468a-9ba7-bbc3bccea539 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:57.112705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70dc6d-b032-468a-9ba7-bbc3bccea539 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:57.112982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70dc6d-b032-468a-9ba7-bbc3bccea539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:57.132046 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-6a70dc6d-b032-468a-9ba7-bbc3bccea539 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:57.133322 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6a70dc6d-b032-468a-9ba7-bbc3bccea539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:57.194113 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1223/2460] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:57.204421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4910fc-2c73-4336-9a88-35f25ab9d43b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:57.206258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4b4910fc-2c73-4336-9a88-35f25ab9d43b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:57.259190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:57.266144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4910fc-2c73-4336-9a88-35f25ab9d43b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:57.270163 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fce73-49c1-4cf2-a8e8-b727b4ab7818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:57.270767 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fce73-49c1-4cf2-a8e8-b727b4ab7818 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:57.271044 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fce73-49c1-4cf2-a8e8-b727b4ab7818 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:57.271373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fce73-49c1-4cf2-a8e8-b727b4ab7818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:57.280273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fce73-49c1-4cf2-a8e8-b727b4ab7818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:57.281732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fce73-49c1-4cf2-a8e8-b727b4ab7818 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:57.291548 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1238/2461] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:32:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:32:57.932743 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb27f24-81c5-42c5-a5f7-47143e921ca7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:57.933060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb27f24-81c5-42c5-a5f7-47143e921ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:57.933342 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb27f24-81c5-42c5-a5f7-47143e921ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:57.966372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-3fb27f24-81c5-42c5-a5f7-47143e921ca7 None None] MFA Rules not processed for user `e989ae02a7d74bb996b24bed89e7459a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:57.967634 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3fb27f24-81c5-42c5-a5f7-47143e921ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:58.039715 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1224/2462] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:57 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:32:58.044173 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fce73-49c1-4cf2-a8e8-b727b4ab7818 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:58.055563 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fce73-49c1-4cf2-a8e8-b727b4ab7818 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:58.061208 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4afc7-b729-49a1-be57-e18e81a6a860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:58.061893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4afc7-b729-49a1-be57-e18e81a6a860 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:58.062173 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4afc7-b729-49a1-be57-e18e81a6a860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:58.062515 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4afc7-b729-49a1-be57-e18e81a6a860 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:58.064008 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4afc7-b729-49a1-be57-e18e81a6a860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:58.065947 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4afc7-b729-49a1-be57-e18e81a6a860 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:58.097078 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1239/2463] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:32:58 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:58.098258 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb27f24-81c5-42c5-a5f7-47143e921ca7 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:58.111548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb27f24-81c5-42c5-a5f7-47143e921ca7 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:58.183443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfb836-9843-47e5-82b1-dd584ea7240c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:58.184077 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfb836-9843-47e5-82b1-dd584ea7240c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:58.184358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfb836-9843-47e5-82b1-dd584ea7240c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:58.184631 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfb836-9843-47e5-82b1-dd584ea7240c admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:58.185309 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfb836-9843-47e5-82b1-dd584ea7240c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:58.187296 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfb836-9843-47e5-82b1-dd584ea7240c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:58.198927 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:32:58.216676 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1225/2464] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:32:58 2022] POST /identity/v3/users => generated 308 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:32:58.217880 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4afc7-b729-49a1-be57-e18e81a6a860 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:58.231551 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4afc7-b729-49a1-be57-e18e81a6a860 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:58.237032 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae942400-e5bf-448b-a1fc-440497f4c749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:58.237738 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae942400-e5bf-448b-a1fc-440497f4c749 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:58.238205 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae942400-e5bf-448b-a1fc-440497f4c749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:58.238570 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae942400-e5bf-448b-a1fc-440497f4c749 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:58.239337 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae942400-e5bf-448b-a1fc-440497f4c749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:58.241771 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae942400-e5bf-448b-a1fc-440497f4c749 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:58.253702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1240/2465] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:32:58.258473 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfb836-9843-47e5-82b1-dd584ea7240c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:58.273027 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfb836-9843-47e5-82b1-dd584ea7240c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:58.277664 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb379b-8fb9-4df7-a2a1-57c0aab943ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:58.278321 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb379b-8fb9-4df7-a2a1-57c0aab943ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:58.278597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb379b-8fb9-4df7-a2a1-57c0aab943ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:58.278868 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb379b-8fb9-4df7-a2a1-57c0aab943ec admin admin] PATH_INFO: `/v3/projects/b9632b42fdbc4a35b43016a0e0d4f77b/users/8d7e0afe8f2b4060b3abe4c78e0ff281/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:58.294231 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb379b-8fb9-4df7-a2a1-57c0aab943ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9632b42fdbc4a35b43016a0e0d4f77b, user_id=8d7e0afe8f2b4060b3abe4c78e0ff281, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:58.297497 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb379b-8fb9-4df7-a2a1-57c0aab943ec admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:58.310548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1226/2466] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:58 2022] PUT /identity/v3/projects/b9632b42fdbc4a35b43016a0e0d4f77b/users/8d7e0afe8f2b4060b3abe4c78e0ff281/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:58.315313 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae942400-e5bf-448b-a1fc-440497f4c749 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:58.325383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae942400-e5bf-448b-a1fc-440497f4c749 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:58.384244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcac551-4622-4ae5-b19b-4b5d09eebc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:58.384759 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcac551-4622-4ae5-b19b-4b5d09eebc2d admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:58.385040 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcac551-4622-4ae5-b19b-4b5d09eebc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:58.385315 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcac551-4622-4ae5-b19b-4b5d09eebc2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:58.385951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcac551-4622-4ae5-b19b-4b5d09eebc2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:58.387609 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcac551-4622-4ae5-b19b-4b5d09eebc2d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:58.397424 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1241/2467] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:32:58.401730 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb379b-8fb9-4df7-a2a1-57c0aab943ec admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:32:58.412751 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb379b-8fb9-4df7-a2a1-57c0aab943ec admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:32:58.417820 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fae76e-c684-449b-a39f-a84da53dc686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:32:58.418580 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fae76e-c684-449b-a39f-a84da53dc686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:58.418868 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fae76e-c684-449b-a39f-a84da53dc686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:58.419178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fae76e-c684-449b-a39f-a84da53dc686 admin admin] PATH_INFO: `/v3/projects/b9632b42fdbc4a35b43016a0e0d4f77b/users/8d7e0afe8f2b4060b3abe4c78e0ff281/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:58.421230 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fae76e-c684-449b-a39f-a84da53dc686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9632b42fdbc4a35b43016a0e0d4f77b, user_id=8d7e0afe8f2b4060b3abe4c78e0ff281, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:32:58.424530 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fae76e-c684-449b-a39f-a84da53dc686 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:32:58.435193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1227/2468] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:32:58 2022] PUT /identity/v3/projects/b9632b42fdbc4a35b43016a0e0d4f77b/users/8d7e0afe8f2b4060b3abe4c78e0ff281/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:32:59.085392 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1200a3-4cc1-4d9c-b17c-8c0b416bcc18 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:32:59.085676 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1200a3-4cc1-4d9c-b17c-8c0b416bcc18 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:32:59.085944 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1200a3-4cc1-4d9c-b17c-8c0b416bcc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:32:59.116470 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-7d1200a3-4cc1-4d9c-b17c-8c0b416bcc18 None None] MFA Rules not processed for user `8d7e0afe8f2b4060b3abe4c78e0ff281`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:32:59.117688 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7d1200a3-4cc1-4d9c-b17c-8c0b416bcc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:32:59.192226 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1242/2469] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:32:59 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:00.498197 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fae76e-c684-449b-a39f-a84da53dc686 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:00.567639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fae76e-c684-449b-a39f-a84da53dc686 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:00.572411 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061f8be-7631-49d4-af47-0b5a8fb75801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:00.573038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061f8be-7631-49d4-af47-0b5a8fb75801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:00.573311 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061f8be-7631-49d4-af47-0b5a8fb75801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:00.573642 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061f8be-7631-49d4-af47-0b5a8fb75801 admin admin] PATH_INFO: `/v3/users/e989ae02a7d74bb996b24bed89e7459a` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:00.574749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061f8be-7631-49d4-af47-0b5a8fb75801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e989ae02a7d74bb996b24bed89e7459a)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:00.576705 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061f8be-7631-49d4-af47-0b5a8fb75801 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:00.622788 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8061f8be-7631-49d4-af47-0b5a8fb75801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e989ae02a7d74bb996b24bed89e7459a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:00.631187 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8061f8be-7631-49d4-af47-0b5a8fb75801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e989ae02a7d74bb996b24bed89e7459a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:00.631790 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8061f8be-7631-49d4-af47-0b5a8fb75801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e989ae02a7d74bb996b24bed89e7459a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:00.644332 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8061f8be-7631-49d4-af47-0b5a8fb75801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e989ae02a7d74bb996b24bed89e7459a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:00.651383 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8061f8be-7631-49d4-af47-0b5a8fb75801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e989ae02a7d74bb996b24bed89e7459a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:00.660125 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1228/2470] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:33:00 2022] DELETE /identity/v3/users/e989ae02a7d74bb996b24bed89e7459a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:00.836556 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1200a3-4cc1-4d9c-b17c-8c0b416bcc18 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:00.850940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1200a3-4cc1-4d9c-b17c-8c0b416bcc18 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:00.894720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:00.918636 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:00.919296 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:00.919575 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:00.919846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] PATH_INFO: `/v3/projects/097f22ab9d854cb39e0a75f632ae1362` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:00.920951 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=097f22ab9d854cb39e0a75f632ae1362)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:00.923485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:00.968101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '097f22ab9d854cb39e0a75f632ae1362', 'request_id': 'req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:00.974321 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '097f22ab9d854cb39e0a75f632ae1362', 'request_id': 'req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:00.980222 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '097f22ab9d854cb39e0a75f632ae1362', 'request_id': 'req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:00.992559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] The token cache is being invalidate because project 097f22ab9d854cb39e0a75f632ae1362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:33:00.992840 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:00.993449 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:00.994905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1243/2471] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:33:00 2022] DELETE /identity/v3/projects/097f22ab9d854cb39e0a75f632ae1362 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:00.998847 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061f8be-7631-49d4-af47-0b5a8fb75801 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:01.000548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8061f8be-7631-49d4-af47-0b5a8fb75801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:01.013468 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061f8be-7631-49d4-af47-0b5a8fb75801 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:01.045760 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:01.074651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b346d-15cc-4f4d-a31a-fb6d429261df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:01.075524 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b346d-15cc-4f4d-a31a-fb6d429261df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:01.075988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b346d-15cc-4f4d-a31a-fb6d429261df admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:01.076387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b346d-15cc-4f4d-a31a-fb6d429261df admin admin] PATH_INFO: `/v3/users/8d7e0afe8f2b4060b3abe4c78e0ff281` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:01.077709 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b346d-15cc-4f4d-a31a-fb6d429261df admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d7e0afe8f2b4060b3abe4c78e0ff281)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:01.080262 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b346d-15cc-4f4d-a31a-fb6d429261df admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:01.128607 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-187b346d-15cc-4f4d-a31a-fb6d429261df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d7e0afe8f2b4060b3abe4c78e0ff281'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:01.135607 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-187b346d-15cc-4f4d-a31a-fb6d429261df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d7e0afe8f2b4060b3abe4c78e0ff281'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:01.136219 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-187b346d-15cc-4f4d-a31a-fb6d429261df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d7e0afe8f2b4060b3abe4c78e0ff281'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:01.149229 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-187b346d-15cc-4f4d-a31a-fb6d429261df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d7e0afe8f2b4060b3abe4c78e0ff281'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:01.156520 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-187b346d-15cc-4f4d-a31a-fb6d429261df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d7e0afe8f2b4060b3abe4c78e0ff281'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:01.162454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1229/2472] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:33:00 2022] DELETE /identity/v3/users/8d7e0afe8f2b4060b3abe4c78e0ff281 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:01.328628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:01.330066 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:01.346487 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc14db-bf20-4a6e-892a-3b169e5aa4f8 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:01.404834 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b4957-4626-46bc-8a5a-4e5949cdf372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:01.405383 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b4957-4626-46bc-8a5a-4e5949cdf372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:01.405660 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b4957-4626-46bc-8a5a-4e5949cdf372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:01.405976 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b4957-4626-46bc-8a5a-4e5949cdf372 admin admin] PATH_INFO: `/v3/projects/b9632b42fdbc4a35b43016a0e0d4f77b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:01.407082 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b4957-4626-46bc-8a5a-4e5949cdf372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9632b42fdbc4a35b43016a0e0d4f77b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:01.408975 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b4957-4626-46bc-8a5a-4e5949cdf372 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:01.454008 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-208b4957-4626-46bc-8a5a-4e5949cdf372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9632b42fdbc4a35b43016a0e0d4f77b', 'request_id': 'req-208b4957-4626-46bc-8a5a-4e5949cdf372'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:01.459443 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-208b4957-4626-46bc-8a5a-4e5949cdf372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9632b42fdbc4a35b43016a0e0d4f77b', 'request_id': 'req-208b4957-4626-46bc-8a5a-4e5949cdf372'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:01.465827 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-208b4957-4626-46bc-8a5a-4e5949cdf372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9632b42fdbc4a35b43016a0e0d4f77b', 'request_id': 'req-208b4957-4626-46bc-8a5a-4e5949cdf372'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:01.474468 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:01.479793 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-208b4957-4626-46bc-8a5a-4e5949cdf372 admin admin] The token cache is being invalidate because project b9632b42fdbc4a35b43016a0e0d4f77b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:33:01.480293 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-208b4957-4626-46bc-8a5a-4e5949cdf372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:01.481679 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-208b4957-4626-46bc-8a5a-4e5949cdf372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:01.483328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1244/2473] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:33:01 2022] DELETE /identity/v3/projects/b9632b42fdbc4a35b43016a0e0d4f77b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:33:01.488304 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe6bfc-bf20-4eba-91b9-ec6405d02cde None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:01.488596 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe6bfc-bf20-4eba-91b9-ec6405d02cde None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:01.488878 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe6bfc-bf20-4eba-91b9-ec6405d02cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:01.508817 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9cfe6bfc-bf20-4eba-91b9-ec6405d02cde None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:01.510094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9cfe6bfc-bf20-4eba-91b9-ec6405d02cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:01.570345 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1230/2474] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:01.579855 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a701c8-704b-40a0-8123-71020211ef40 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:01.580133 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a701c8-704b-40a0-8123-71020211ef40 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:01.580400 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a701c8-704b-40a0-8123-71020211ef40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:01.599786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-33a701c8-704b-40a0-8123-71020211ef40 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:01.600991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-33a701c8-704b-40a0-8123-71020211ef40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:01.609312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1245/2475] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:01.613502 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe6bfc-bf20-4eba-91b9-ec6405d02cde None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:01.623268 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe6bfc-bf20-4eba-91b9-ec6405d02cde None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:01.628587 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fce1b2-9bf8-40e8-a039-f2a5b7457702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:01.629193 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fce1b2-9bf8-40e8-a039-f2a5b7457702 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:01.629471 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fce1b2-9bf8-40e8-a039-f2a5b7457702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:01.629742 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fce1b2-9bf8-40e8-a039-f2a5b7457702 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:01.630391 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fce1b2-9bf8-40e8-a039-f2a5b7457702 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:33:01.630696 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fce1b2-9bf8-40e8-a039-f2a5b7457702 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:01.632336 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fce1b2-9bf8-40e8-a039-f2a5b7457702 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:01.644255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1231/2476] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:33:01.648322 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a701c8-704b-40a0-8123-71020211ef40 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:01.657733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a701c8-704b-40a0-8123-71020211ef40 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:01.662250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842458cd-ffd7-4e59-8c68-7bf145721357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:01.662873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842458cd-ffd7-4e59-8c68-7bf145721357 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:01.663209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842458cd-ffd7-4e59-8c68-7bf145721357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:01.663505 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842458cd-ffd7-4e59-8c68-7bf145721357 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:01.664172 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842458cd-ffd7-4e59-8c68-7bf145721357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:01.666082 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842458cd-ffd7-4e59-8c68-7bf145721357 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:01.694413 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1246/2477] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:33:01 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:33:01.697825 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fce1b2-9bf8-40e8-a039-f2a5b7457702 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:01.708099 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fce1b2-9bf8-40e8-a039-f2a5b7457702 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:01.757902 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:01.776023 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e305b-9bd7-486b-a62f-a611e63451d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:01.776634 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e305b-9bd7-486b-a62f-a611e63451d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:01.776913 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e305b-9bd7-486b-a62f-a611e63451d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:01.777185 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e305b-9bd7-486b-a62f-a611e63451d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:01.777840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e305b-9bd7-486b-a62f-a611e63451d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:01.779821 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e305b-9bd7-486b-a62f-a611e63451d1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:01.803392 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1232/2478] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:33:01 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:33:01.807592 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842458cd-ffd7-4e59-8c68-7bf145721357 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:01.818432 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842458cd-ffd7-4e59-8c68-7bf145721357 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:01.822913 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f61536-e055-40f1-a6bd-ffc7821dbebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:01.823539 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f61536-e055-40f1-a6bd-ffc7821dbebb admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:01.823842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f61536-e055-40f1-a6bd-ffc7821dbebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:01.824108 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f61536-e055-40f1-a6bd-ffc7821dbebb admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:01.824736 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f61536-e055-40f1-a6bd-ffc7821dbebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:01.826311 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f61536-e055-40f1-a6bd-ffc7821dbebb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:01.836258 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1247/2479] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:33:01.840256 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e305b-9bd7-486b-a62f-a611e63451d1 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:01.850279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e305b-9bd7-486b-a62f-a611e63451d1 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:01.854812 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42cbea-648a-4a43-bd9b-9598179b5cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:01.855719 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42cbea-648a-4a43-bd9b-9598179b5cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:01.855996 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42cbea-648a-4a43-bd9b-9598179b5cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:01.856264 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42cbea-648a-4a43-bd9b-9598179b5cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:01.856888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42cbea-648a-4a43-bd9b-9598179b5cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:01.858554 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42cbea-648a-4a43-bd9b-9598179b5cb2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:01.869744 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1233/2480] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:33:01.874219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f61536-e055-40f1-a6bd-ffc7821dbebb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:01.885006 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f61536-e055-40f1-a6bd-ffc7821dbebb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:01.889884 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e8dd3-747c-43c6-874d-a1cae56443eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:01.890531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e8dd3-747c-43c6-874d-a1cae56443eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:01.890807 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e8dd3-747c-43c6-874d-a1cae56443eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:01.891101 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e8dd3-747c-43c6-874d-a1cae56443eb admin admin] PATH_INFO: `/v3/projects/e53be85a2ecd4a279464a11ccefeeb13/users/2faa3d28fe34492fa3560b8d786ae9c7/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:01.905269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e8dd3-747c-43c6-874d-a1cae56443eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e53be85a2ecd4a279464a11ccefeeb13, user_id=2faa3d28fe34492fa3560b8d786ae9c7, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:01.908225 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e8dd3-747c-43c6-874d-a1cae56443eb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:01.920784 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1248/2481] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:33:01 2022] PUT /identity/v3/projects/e53be85a2ecd4a279464a11ccefeeb13/users/2faa3d28fe34492fa3560b8d786ae9c7/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:01.931753 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7212d46-d196-4494-ae0d-7c22dbb771b8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:01.932040 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7212d46-d196-4494-ae0d-7c22dbb771b8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:01.932312 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7212d46-d196-4494-ae0d-7c22dbb771b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:01.961613 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e7212d46-d196-4494-ae0d-7c22dbb771b8 None None] MFA Rules not processed for user `2faa3d28fe34492fa3560b8d786ae9c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:01.962934 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e7212d46-d196-4494-ae0d-7c22dbb771b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:02.025610 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1234/2482] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:02.030174 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e8dd3-747c-43c6-874d-a1cae56443eb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:02.090003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e8dd3-747c-43c6-874d-a1cae56443eb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:02.095313 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e153e-d4fa-4734-a6e6-d0e2d8bb888e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:02.095910 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e153e-d4fa-4734-a6e6-d0e2d8bb888e admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:02.096185 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e153e-d4fa-4734-a6e6-d0e2d8bb888e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:02.096453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e153e-d4fa-4734-a6e6-d0e2d8bb888e admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:02.097070 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e153e-d4fa-4734-a6e6-d0e2d8bb888e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:02.098948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e153e-d4fa-4734-a6e6-d0e2d8bb888e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:02.125153 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1249/2483] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:33:02 2022] POST /identity/v3/projects => generated 393 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:33:02.129372 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7212d46-d196-4494-ae0d-7c22dbb771b8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:02.140152 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7212d46-d196-4494-ae0d-7c22dbb771b8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:02.195044 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d591acd-705b-4af3-a7ee-722351b318b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:02.195627 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d591acd-705b-4af3-a7ee-722351b318b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:02.195901 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d591acd-705b-4af3-a7ee-722351b318b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:02.196171 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d591acd-705b-4af3-a7ee-722351b318b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:02.196785 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d591acd-705b-4af3-a7ee-722351b318b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:02.198698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d591acd-705b-4af3-a7ee-722351b318b3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:02.219530 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1235/2484] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:33:02 2022] POST /identity/v3/users => generated 303 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:33:02.223640 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e153e-d4fa-4734-a6e6-d0e2d8bb888e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:02.234661 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e153e-d4fa-4734-a6e6-d0e2d8bb888e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:02.239577 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a194a2a-c5c5-4451-a120-5abc988b9f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:02.240169 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a194a2a-c5c5-4451-a120-5abc988b9f72 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:02.240450 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a194a2a-c5c5-4451-a120-5abc988b9f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:02.240722 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a194a2a-c5c5-4451-a120-5abc988b9f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:02.241359 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a194a2a-c5c5-4451-a120-5abc988b9f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:02.242957 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a194a2a-c5c5-4451-a120-5abc988b9f72 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:02.253632 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1250/2485] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:33:02.257653 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d591acd-705b-4af3-a7ee-722351b318b3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:02.267550 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d591acd-705b-4af3-a7ee-722351b318b3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:02.272449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62a2fb-a6cc-4dea-9509-56bf8459b170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:02.273098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62a2fb-a6cc-4dea-9509-56bf8459b170 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:02.273390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62a2fb-a6cc-4dea-9509-56bf8459b170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:02.273646 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62a2fb-a6cc-4dea-9509-56bf8459b170 admin admin] PATH_INFO: `/v3/projects/cbe7e598d3ad400a9cecf30b29c4ee2c/users/dc285f266573462ba35f4c0b55846a8a/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:02.288778 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62a2fb-a6cc-4dea-9509-56bf8459b170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbe7e598d3ad400a9cecf30b29c4ee2c, user_id=dc285f266573462ba35f4c0b55846a8a, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:02.291501 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62a2fb-a6cc-4dea-9509-56bf8459b170 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:02.306263 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1236/2486] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:33:02 2022] PUT /identity/v3/projects/cbe7e598d3ad400a9cecf30b29c4ee2c/users/dc285f266573462ba35f4c0b55846a8a/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:02.310151 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a194a2a-c5c5-4451-a120-5abc988b9f72 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:02.320436 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a194a2a-c5c5-4451-a120-5abc988b9f72 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:02.375908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3eb2d-b711-43c4-a5af-a7844a5fe4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:02.376853 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3eb2d-b711-43c4-a5af-a7844a5fe4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:02.377299 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3eb2d-b711-43c4-a5af-a7844a5fe4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:02.377609 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3eb2d-b711-43c4-a5af-a7844a5fe4d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:02.378246 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3eb2d-b711-43c4-a5af-a7844a5fe4d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:02.379864 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3eb2d-b711-43c4-a5af-a7844a5fe4d0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:02.389434 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1251/2487] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:33:02.393563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62a2fb-a6cc-4dea-9509-56bf8459b170 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:02.403590 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62a2fb-a6cc-4dea-9509-56bf8459b170 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:02.408631 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a04a2-b7bc-49d6-99f6-f2b10e550df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:02.409275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a04a2-b7bc-49d6-99f6-f2b10e550df4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:02.409551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a04a2-b7bc-49d6-99f6-f2b10e550df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:02.409823 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a04a2-b7bc-49d6-99f6-f2b10e550df4 admin admin] PATH_INFO: `/v3/projects/cbe7e598d3ad400a9cecf30b29c4ee2c/users/dc285f266573462ba35f4c0b55846a8a/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:02.411730 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a04a2-b7bc-49d6-99f6-f2b10e550df4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbe7e598d3ad400a9cecf30b29c4ee2c, user_id=dc285f266573462ba35f4c0b55846a8a, role_id=9e3f7aa212fd4b3ea4d1fabd0de37a4b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:02.414339 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a04a2-b7bc-49d6-99f6-f2b10e550df4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:02.424900 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1237/2488] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:33:02 2022] PUT /identity/v3/projects/cbe7e598d3ad400a9cecf30b29c4ee2c/users/dc285f266573462ba35f4c0b55846a8a/roles/9e3f7aa212fd4b3ea4d1fabd0de37a4b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:02.434715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c971999-db88-407e-ba95-aac32a3ebe33 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:02.434999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c971999-db88-407e-ba95-aac32a3ebe33 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:02.435307 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c971999-db88-407e-ba95-aac32a3ebe33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:02.443373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:02.465715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8c971999-db88-407e-ba95-aac32a3ebe33 None None] MFA Rules not processed for user `dc285f266573462ba35f4c0b55846a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:02.466832 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8c971999-db88-407e-ba95-aac32a3ebe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:02.535456 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1252/2489] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:02.546599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a04a2-b7bc-49d6-99f6-f2b10e550df4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:02.548470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-749a04a2-b7bc-49d6-99f6-f2b10e550df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:02.552303 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:02.628070 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a04a2-b7bc-49d6-99f6-f2b10e550df4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:02.631875 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d46ca3-2eb2-4601-8862-22d816c5e007 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:02.632481 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d46ca3-2eb2-4601-8862-22d816c5e007 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:02.632757 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d46ca3-2eb2-4601-8862-22d816c5e007 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:02.633024 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d46ca3-2eb2-4601-8862-22d816c5e007 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:02.642833 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d46ca3-2eb2-4601-8862-22d816c5e007 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:02.644307 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d46ca3-2eb2-4601-8862-22d816c5e007 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:02.654364 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1238/2490] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:33:02 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:33:02.699962 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c971999-db88-407e-ba95-aac32a3ebe33 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:02.701344 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8c971999-db88-407e-ba95-aac32a3ebe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:02.713992 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c971999-db88-407e-ba95-aac32a3ebe33 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:02.717642 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3968b-43c3-4509-94cc-272f8192393b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:02.718180 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3968b-43c3-4509-94cc-272f8192393b service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:02.719091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3968b-43c3-4509-94cc-272f8192393b service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:02.719377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3968b-43c3-4509-94cc-272f8192393b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:02.776193 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3968b-43c3-4509-94cc-272f8192393b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:02.777649 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3968b-43c3-4509-94cc-272f8192393b service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:02.787701 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1253/2491] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:33:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:33:06.072278 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d46ca3-2eb2-4601-8862-22d816c5e007 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:06.135859 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d46ca3-2eb2-4601-8862-22d816c5e007 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:06.140420 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd57b4-6ca5-4264-a75f-7c85d378192b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:06.141047 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd57b4-6ca5-4264-a75f-7c85d378192b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:06.141326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd57b4-6ca5-4264-a75f-7c85d378192b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:06.142141 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd57b4-6ca5-4264-a75f-7c85d378192b admin admin] PATH_INFO: `/v3/users/2faa3d28fe34492fa3560b8d786ae9c7` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:06.143284 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd57b4-6ca5-4264-a75f-7c85d378192b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2faa3d28fe34492fa3560b8d786ae9c7)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:06.145201 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd57b4-6ca5-4264-a75f-7c85d378192b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:06.193060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d8cd57b4-6ca5-4264-a75f-7c85d378192b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2faa3d28fe34492fa3560b8d786ae9c7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.201435 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d8cd57b4-6ca5-4264-a75f-7c85d378192b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2faa3d28fe34492fa3560b8d786ae9c7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.204883 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d8cd57b4-6ca5-4264-a75f-7c85d378192b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2faa3d28fe34492fa3560b8d786ae9c7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.216723 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d8cd57b4-6ca5-4264-a75f-7c85d378192b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2faa3d28fe34492fa3560b8d786ae9c7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.223565 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d8cd57b4-6ca5-4264-a75f-7c85d378192b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2faa3d28fe34492fa3560b8d786ae9c7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.229488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1239/2492] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:33:06 2022] DELETE /identity/v3/users/2faa3d28fe34492fa3560b8d786ae9c7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:33:06.234298 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152425dd-0c0f-4d63-96f2-475906428851 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:06.234775 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152425dd-0c0f-4d63-96f2-475906428851 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:06.234883 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152425dd-0c0f-4d63-96f2-475906428851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:06.255786 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-152425dd-0c0f-4d63-96f2-475906428851 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:06.257080 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-152425dd-0c0f-4d63-96f2-475906428851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:06.322352 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1254/2493] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:33:06.332604 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd57b4-6ca5-4264-a75f-7c85d378192b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:06.334021 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d8cd57b4-6ca5-4264-a75f-7c85d378192b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:06.340560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:06.394503 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd57b4-6ca5-4264-a75f-7c85d378192b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:06.398104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d88b9-00f8-4a56-89d4-07b71921f547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:06.398716 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d88b9-00f8-4a56-89d4-07b71921f547 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:06.399019 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d88b9-00f8-4a56-89d4-07b71921f547 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:06.399330 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d88b9-00f8-4a56-89d4-07b71921f547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:06.409101 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d88b9-00f8-4a56-89d4-07b71921f547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:06.410544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d88b9-00f8-4a56-89d4-07b71921f547 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:06.420015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1240/2494] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:33:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:33:06.698444 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152425dd-0c0f-4d63-96f2-475906428851 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:06.699882 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-152425dd-0c0f-4d63-96f2-475906428851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:06.713122 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152425dd-0c0f-4d63-96f2-475906428851 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:06.718150 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:06.718964 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:06.719275 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:06.720021 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] PATH_INFO: `/v3/projects/e53be85a2ecd4a279464a11ccefeeb13` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:06.721144 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e53be85a2ecd4a279464a11ccefeeb13)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:06.723039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:06.749184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:06.771273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e53be85a2ecd4a279464a11ccefeeb13', 'request_id': 'req-39f04eef-3039-47d9-a565-577cb91057fe'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.777323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e53be85a2ecd4a279464a11ccefeeb13', 'request_id': 'req-39f04eef-3039-47d9-a565-577cb91057fe'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.784030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e53be85a2ecd4a279464a11ccefeeb13', 'request_id': 'req-39f04eef-3039-47d9-a565-577cb91057fe'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.795807 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] The token cache is being invalidate because project e53be85a2ecd4a279464a11ccefeeb13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:33:06.796104 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.796724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.798784 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1255/2495] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:33:06 2022] DELETE /identity/v3/projects/e53be85a2ecd4a279464a11ccefeeb13 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:06.802918 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d88b9-00f8-4a56-89d4-07b71921f547 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:06.812433 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d88b9-00f8-4a56-89d4-07b71921f547 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:06.867917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baa166-6c9a-4c14-960d-327f40873e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:06.868533 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baa166-6c9a-4c14-960d-327f40873e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:06.868817 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baa166-6c9a-4c14-960d-327f40873e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:06.869092 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baa166-6c9a-4c14-960d-327f40873e16 admin admin] PATH_INFO: `/v3/users/dc285f266573462ba35f4c0b55846a8a` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:06.870212 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baa166-6c9a-4c14-960d-327f40873e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc285f266573462ba35f4c0b55846a8a)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:06.872186 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baa166-6c9a-4c14-960d-327f40873e16 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:06.919293 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d0baa166-6c9a-4c14-960d-327f40873e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc285f266573462ba35f4c0b55846a8a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.926509 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d0baa166-6c9a-4c14-960d-327f40873e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc285f266573462ba35f4c0b55846a8a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.927145 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d0baa166-6c9a-4c14-960d-327f40873e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc285f266573462ba35f4c0b55846a8a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.938792 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d0baa166-6c9a-4c14-960d-327f40873e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc285f266573462ba35f4c0b55846a8a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.945671 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d0baa166-6c9a-4c14-960d-327f40873e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc285f266573462ba35f4c0b55846a8a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:06.951512 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1241/2496] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:33:06 2022] DELETE /identity/v3/users/dc285f266573462ba35f4c0b55846a8a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:07.213376 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:07.214909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:07.229319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f04eef-3039-47d9-a565-577cb91057fe admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:07.288319 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:07.288974 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:07.289216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:07.289486 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] PATH_INFO: `/v3/projects/cbe7e598d3ad400a9cecf30b29c4ee2c` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:07.290598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbe7e598d3ad400a9cecf30b29c4ee2c)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:07.292537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:07.337730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbe7e598d3ad400a9cecf30b29c4ee2c', 'request_id': 'req-5a0ab195-03fa-4cff-853a-042896449b79'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:07.343488 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbe7e598d3ad400a9cecf30b29c4ee2c', 'request_id': 'req-5a0ab195-03fa-4cff-853a-042896449b79'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:07.350010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbe7e598d3ad400a9cecf30b29c4ee2c', 'request_id': 'req-5a0ab195-03fa-4cff-853a-042896449b79'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:07.363785 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] The token cache is being invalidate because project cbe7e598d3ad400a9cecf30b29c4ee2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:33:07.364071 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:07.364720 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:07.366417 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1256/2497] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:33:07 2022] DELETE /identity/v3/projects/cbe7e598d3ad400a9cecf30b29c4ee2c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:07.371977 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4579d2b-ac78-4bdc-8db6-fa9180da6e95 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:07.372441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4579d2b-ac78-4bdc-8db6-fa9180da6e95 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:07.372540 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4579d2b-ac78-4bdc-8db6-fa9180da6e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:07.383176 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:07.384603 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:07.397176 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e4579d2b-ac78-4bdc-8db6-fa9180da6e95 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:07.398366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e4579d2b-ac78-4bdc-8db6-fa9180da6e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:07.418326 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:07.432550 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:07.454711 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ab195-03fa-4cff-853a-042896449b79 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:07.459292 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097099f-7c98-4593-9c6e-17d9746e0c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:07.459915 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097099f-7c98-4593-9c6e-17d9746e0c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:07.460190 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097099f-7c98-4593-9c6e-17d9746e0c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:07.460461 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097099f-7c98-4593-9c6e-17d9746e0c6a admin admin] PATH_INFO: `/v3/users/84d6928ba28642858d870450eded6ef0` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:07.461572 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097099f-7c98-4593-9c6e-17d9746e0c6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d6928ba28642858d870450eded6ef0)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:07.462702 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1242/2498] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:07.463523 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097099f-7c98-4593-9c6e-17d9746e0c6a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:07.471214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7520238-6305-42ca-b969-c81bfadf9a96 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:07.471500 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7520238-6305-42ca-b969-c81bfadf9a96 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:07.471775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7520238-6305-42ca-b969-c81bfadf9a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:07.493276 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b7520238-6305-42ca-b969-c81bfadf9a96 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:07.494591 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b7520238-6305-42ca-b969-c81bfadf9a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:07.503982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1243/2499] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:33:07.508188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7520238-6305-42ca-b969-c81bfadf9a96 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:07.510579 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6097099f-7c98-4593-9c6e-17d9746e0c6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d6928ba28642858d870450eded6ef0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:07.518179 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7520238-6305-42ca-b969-c81bfadf9a96 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:07.521459 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6097099f-7c98-4593-9c6e-17d9746e0c6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d6928ba28642858d870450eded6ef0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:07.523685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca4367-d8d4-4007-a16f-3681bf0afc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:07.524011 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca4367-d8d4-4007-a16f-3681bf0afc9d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:07.524279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca4367-d8d4-4007-a16f-3681bf0afc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:07.524535 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca4367-d8d4-4007-a16f-3681bf0afc9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:07.525175 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca4367-d8d4-4007-a16f-3681bf0afc9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:33:07.525475 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca4367-d8d4-4007-a16f-3681bf0afc9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:07.527317 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca4367-d8d4-4007-a16f-3681bf0afc9d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:07.535648 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6097099f-7c98-4593-9c6e-17d9746e0c6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d6928ba28642858d870450eded6ef0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:07.540234 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1244/2500] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:33:07.544436 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca4367-d8d4-4007-a16f-3681bf0afc9d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:07.546453 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6097099f-7c98-4593-9c6e-17d9746e0c6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d6928ba28642858d870450eded6ef0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:07.547185 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6097099f-7c98-4593-9c6e-17d9746e0c6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d6928ba28642858d870450eded6ef0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:07.554872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca4367-d8d4-4007-a16f-3681bf0afc9d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:07.557353 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1257/2501] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:33:07 2022] DELETE /identity/v3/users/84d6928ba28642858d870450eded6ef0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:33:07.559602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa9d00-a6be-44fd-b02e-3cdfae036a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:07.560207 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa9d00-a6be-44fd-b02e-3cdfae036a7a admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:07.560484 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa9d00-a6be-44fd-b02e-3cdfae036a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:07.560755 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa9d00-a6be-44fd-b02e-3cdfae036a7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:07.561369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa9d00-a6be-44fd-b02e-3cdfae036a7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:07.562112 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71b065-6ff6-442b-b352-91596d057fd7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:07.562469 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71b065-6ff6-442b-b352-91596d057fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:07.562747 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71b065-6ff6-442b-b352-91596d057fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:07.563985 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa9d00-a6be-44fd-b02e-3cdfae036a7a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:07.585341 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0d71b065-6ff6-442b-b352-91596d057fd7 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:07.586531 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0d71b065-6ff6-442b-b352-91596d057fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:07.591701 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1245/2502] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:33:07 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:33:07.596260 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa9d00-a6be-44fd-b02e-3cdfae036a7a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:07.597815 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2aaa9d00-a6be-44fd-b02e-3cdfae036a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:07.651240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1258/2503] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:33:07.660989 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71b065-6ff6-442b-b352-91596d057fd7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:07.661498 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa9d00-a6be-44fd-b02e-3cdfae036a7a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:07.662512 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0d71b065-6ff6-442b-b352-91596d057fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:07.666062 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd91b4-9fd4-411f-a825-b82b1749626e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:07.666672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd91b4-9fd4-411f-a825-b82b1749626e admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:07.666947 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd91b4-9fd4-411f-a825-b82b1749626e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:07.667255 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd91b4-9fd4-411f-a825-b82b1749626e admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:07.667878 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd91b4-9fd4-411f-a825-b82b1749626e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:07.669927 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd91b4-9fd4-411f-a825-b82b1749626e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:07.691802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1246/2504] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:33:07 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:33:07.695670 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd91b4-9fd4-411f-a825-b82b1749626e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:07.705604 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd91b4-9fd4-411f-a825-b82b1749626e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:07.710802 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e26d92-aff0-4d25-8631-4c2998ef7931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:07.711601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e26d92-aff0-4d25-8631-4c2998ef7931 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:07.712139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e26d92-aff0-4d25-8631-4c2998ef7931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:07.712139 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e26d92-aff0-4d25-8631-4c2998ef7931 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:07.712785 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e26d92-aff0-4d25-8631-4c2998ef7931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:07.714376 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e26d92-aff0-4d25-8631-4c2998ef7931 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:07.723288 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71b065-6ff6-442b-b352-91596d057fd7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:07.724546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1247/2505] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:33:07.728434 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e26d92-aff0-4d25-8631-4c2998ef7931 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:07.738203 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:07.744089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e26d92-aff0-4d25-8631-4c2998ef7931 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:07.749284 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365cd9a-e2eb-49df-988c-f5d77c6a1f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:07.750025 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365cd9a-e2eb-49df-988c-f5d77c6a1f5f admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:07.750305 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365cd9a-e2eb-49df-988c-f5d77c6a1f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:07.750600 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365cd9a-e2eb-49df-988c-f5d77c6a1f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:07.752514 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365cd9a-e2eb-49df-988c-f5d77c6a1f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:07.754209 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365cd9a-e2eb-49df-988c-f5d77c6a1f5f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:07.764880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1248/2506] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:33:07.769501 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365cd9a-e2eb-49df-988c-f5d77c6a1f5f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:07.779566 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365cd9a-e2eb-49df-988c-f5d77c6a1f5f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:07.780070 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366c64b-0fdd-4f7e-84f8-5cd91d8d475d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:07.780732 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366c64b-0fdd-4f7e-84f8-5cd91d8d475d service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:07.781070 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366c64b-0fdd-4f7e-84f8-5cd91d8d475d service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:07.781313 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366c64b-0fdd-4f7e-84f8-5cd91d8d475d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:07.785289 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d14e1-9467-4c50-bc39-6e46b4ab410d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:07.785940 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d14e1-9467-4c50-bc39-6e46b4ab410d admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:07.786218 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d14e1-9467-4c50-bc39-6e46b4ab410d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:07.786493 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d14e1-9467-4c50-bc39-6e46b4ab410d admin admin] PATH_INFO: `/v3/projects/74c82e7c15074f45a70ca8260f4a67d5/users/70df039b13b044969ffbf585850dac01/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:07.791782 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366c64b-0fdd-4f7e-84f8-5cd91d8d475d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:07.793229 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366c64b-0fdd-4f7e-84f8-5cd91d8d475d service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:07.801449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d14e1-9467-4c50-bc39-6e46b4ab410d admin admin] RBAC: Authorizing `identity:create_grant(project_id=74c82e7c15074f45a70ca8260f4a67d5, user_id=70df039b13b044969ffbf585850dac01, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:07.803609 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1259/2507] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:33:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:33:07.804446 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d14e1-9467-4c50-bc39-6e46b4ab410d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:07.818245 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1249/2508] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:33:07 2022] PUT /identity/v3/projects/74c82e7c15074f45a70ca8260f4a67d5/users/70df039b13b044969ffbf585850dac01/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:33:07.829395 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaa0d1-f952-449f-92ff-b4496ccf5b98 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:07.829577 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaa0d1-f952-449f-92ff-b4496ccf5b98 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:07.829984 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaa0d1-f952-449f-92ff-b4496ccf5b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:07.858472 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2ccaa0d1-f952-449f-92ff-b4496ccf5b98 None None] MFA Rules not processed for user `70df039b13b044969ffbf585850dac01`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:07.859653 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2ccaa0d1-f952-449f-92ff-b4496ccf5b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:07.926971 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1260/2509] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:07.939667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d14e1-9467-4c50-bc39-6e46b4ab410d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:07.941413 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-950d14e1-9467-4c50-bc39-6e46b4ab410d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:07.977973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:08.006692 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d14e1-9467-4c50-bc39-6e46b4ab410d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:08.010818 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14650f0-3512-49b0-8de5-9c5cdf9c0d3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c05a1b303c2418fb2941b8ef56f8972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:08.011464 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14650f0-3512-49b0-8de5-9c5cdf9c0d3c service glance] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:08.011761 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14650f0-3512-49b0-8de5-9c5cdf9c0d3c service glance] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:08.012121 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14650f0-3512-49b0-8de5-9c5cdf9c0d3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:08.021025 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14650f0-3512-49b0-8de5-9c5cdf9c0d3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:08.022453 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14650f0-3512-49b0-8de5-9c5cdf9c0d3c service glance] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:08.032520 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1250/2510] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:33:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:33:08.088976 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccaa0d1-f952-449f-92ff-b4496ccf5b98 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:08.090576 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2ccaa0d1-f952-449f-92ff-b4496ccf5b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:08.172464 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccaa0d1-f952-449f-92ff-b4496ccf5b98 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:08.177173 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8244bd4-01ad-4edb-9a85-798f7f5a1646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:08.177803 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8244bd4-01ad-4edb-9a85-798f7f5a1646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:08.178080 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8244bd4-01ad-4edb-9a85-798f7f5a1646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:08.178348 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8244bd4-01ad-4edb-9a85-798f7f5a1646 admin admin] PATH_INFO: `/v3/projects/304e0188d24a4b79846b2b63adc7b847` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:08.179492 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8244bd4-01ad-4edb-9a85-798f7f5a1646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=304e0188d24a4b79846b2b63adc7b847)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:08.181386 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8244bd4-01ad-4edb-9a85-798f7f5a1646 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:08.230392 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-e8244bd4-01ad-4edb-9a85-798f7f5a1646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '304e0188d24a4b79846b2b63adc7b847', 'request_id': 'req-e8244bd4-01ad-4edb-9a85-798f7f5a1646'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:08.232355 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:08.236546 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-e8244bd4-01ad-4edb-9a85-798f7f5a1646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '304e0188d24a4b79846b2b63adc7b847', 'request_id': 'req-e8244bd4-01ad-4edb-9a85-798f7f5a1646'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:08.243969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-e8244bd4-01ad-4edb-9a85-798f7f5a1646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '304e0188d24a4b79846b2b63adc7b847', 'request_id': 'req-e8244bd4-01ad-4edb-9a85-798f7f5a1646'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:08.257371 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-e8244bd4-01ad-4edb-9a85-798f7f5a1646 admin admin] The token cache is being invalidate because project 304e0188d24a4b79846b2b63adc7b847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:33:08.257508 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-e8244bd4-01ad-4edb-9a85-798f7f5a1646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:08.258086 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-e8244bd4-01ad-4edb-9a85-798f7f5a1646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:08.259698 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1261/2511] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:33:08 2022] DELETE /identity/v3/projects/304e0188d24a4b79846b2b63adc7b847 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:08.264526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430532dc-8111-4e1f-8895-a982ddb13ddf None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:08.264813 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430532dc-8111-4e1f-8895-a982ddb13ddf None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:08.265087 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430532dc-8111-4e1f-8895-a982ddb13ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:08.286103 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-430532dc-8111-4e1f-8895-a982ddb13ddf None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:08.287333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-430532dc-8111-4e1f-8895-a982ddb13ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:08.350374 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1251/2512] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:08.358381 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d312d1b-b9a7-4d1a-89ff-0ee037ac32fd None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:08.358893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d312d1b-b9a7-4d1a-89ff-0ee037ac32fd None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:08.358998 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d312d1b-b9a7-4d1a-89ff-0ee037ac32fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:08.379514 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8d312d1b-b9a7-4d1a-89ff-0ee037ac32fd None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:08.380713 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8d312d1b-b9a7-4d1a-89ff-0ee037ac32fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:08.389249 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1262/2513] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:08.394371 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430532dc-8111-4e1f-8895-a982ddb13ddf None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:08.404748 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430532dc-8111-4e1f-8895-a982ddb13ddf None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:08.409390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d84c1-eee0-45da-b094-b57ebbb7d27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:08.409995 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d84c1-eee0-45da-b094-b57ebbb7d27f admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:08.410270 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d84c1-eee0-45da-b094-b57ebbb7d27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:08.410537 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d84c1-eee0-45da-b094-b57ebbb7d27f admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:08.411215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d84c1-eee0-45da-b094-b57ebbb7d27f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:33:08.411518 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d84c1-eee0-45da-b094-b57ebbb7d27f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:08.413129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d84c1-eee0-45da-b094-b57ebbb7d27f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:08.426096 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1252/2514] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:33:08.435425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d312d1b-b9a7-4d1a-89ff-0ee037ac32fd None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:08.447758 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d312d1b-b9a7-4d1a-89ff-0ee037ac32fd None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:08.452909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eaaa1-4b05-4f8e-846b-c2a2d6e1140c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:08.453589 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eaaa1-4b05-4f8e-846b-c2a2d6e1140c admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:08.454504 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eaaa1-4b05-4f8e-846b-c2a2d6e1140c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:08.454846 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eaaa1-4b05-4f8e-846b-c2a2d6e1140c admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:08.455711 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eaaa1-4b05-4f8e-846b-c2a2d6e1140c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:08.457659 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eaaa1-4b05-4f8e-846b-c2a2d6e1140c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:08.495656 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1263/2515] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:33:08 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:33:08.499848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d84c1-eee0-45da-b094-b57ebbb7d27f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:08.509945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d84c1-eee0-45da-b094-b57ebbb7d27f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:08.568702 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4b803-f8ec-4986-81c5-c3a8c3ca5ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:08.569320 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4b803-f8ec-4986-81c5-c3a8c3ca5ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:08.569603 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4b803-f8ec-4986-81c5-c3a8c3ca5ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:08.569876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4b803-f8ec-4986-81c5-c3a8c3ca5ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:08.570519 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4b803-f8ec-4986-81c5-c3a8c3ca5ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:08.572470 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4b803-f8ec-4986-81c5-c3a8c3ca5ec7 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:08.593525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1253/2516] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:33:08 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:33:08.598468 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eaaa1-4b05-4f8e-846b-c2a2d6e1140c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:08.609113 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eaaa1-4b05-4f8e-846b-c2a2d6e1140c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:08.614039 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03241ba8-97f1-4579-aff8-8b8889f60c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:08.614641 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03241ba8-97f1-4579-aff8-8b8889f60c8f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:08.614919 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03241ba8-97f1-4579-aff8-8b8889f60c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:08.615230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03241ba8-97f1-4579-aff8-8b8889f60c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:08.615905 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03241ba8-97f1-4579-aff8-8b8889f60c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:08.617488 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03241ba8-97f1-4579-aff8-8b8889f60c8f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:08.628240 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1264/2517] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:33:08.632460 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4b803-f8ec-4986-81c5-c3a8c3ca5ec7 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:08.643486 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4b803-f8ec-4986-81c5-c3a8c3ca5ec7 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:08.648188 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf94e8-2cf2-4a4a-8355-953f7d882b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:08.648791 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf94e8-2cf2-4a4a-8355-953f7d882b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:08.649072 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf94e8-2cf2-4a4a-8355-953f7d882b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:08.649345 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf94e8-2cf2-4a4a-8355-953f7d882b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:08.650031 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf94e8-2cf2-4a4a-8355-953f7d882b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:08.651649 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf94e8-2cf2-4a4a-8355-953f7d882b1b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:08.662984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1254/2518] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:33:08.667373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03241ba8-97f1-4579-aff8-8b8889f60c8f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:08.677491 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03241ba8-97f1-4579-aff8-8b8889f60c8f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:08.682628 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba0985-caff-4c18-b6e0-d15bb44d4292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:08.683305 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba0985-caff-4c18-b6e0-d15bb44d4292 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:08.683581 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba0985-caff-4c18-b6e0-d15bb44d4292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:08.683854 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba0985-caff-4c18-b6e0-d15bb44d4292 admin admin] PATH_INFO: `/v3/projects/d50887bbbe1d4a66bdae02e0360129fb/users/73f3cd4c20404c99b53da6cb2bc33e84/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:08.698268 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba0985-caff-4c18-b6e0-d15bb44d4292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50887bbbe1d4a66bdae02e0360129fb, user_id=73f3cd4c20404c99b53da6cb2bc33e84, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:08.700933 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba0985-caff-4c18-b6e0-d15bb44d4292 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:08.713985 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1265/2519] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:33:08 2022] PUT /identity/v3/projects/d50887bbbe1d4a66bdae02e0360129fb/users/73f3cd4c20404c99b53da6cb2bc33e84/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:08.719001 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b00b31-b897-4bc9-9108-d23516e4ce28 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:08.719338 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b00b31-b897-4bc9-9108-d23516e4ce28 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:08.720136 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b00b31-b897-4bc9-9108-d23516e4ce28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:08.747456 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-91b00b31-b897-4bc9-9108-d23516e4ce28 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:08.748719 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-91b00b31-b897-4bc9-9108-d23516e4ce28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:08.785982 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:08.817156 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1255/2520] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:08.826860 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba0985-caff-4c18-b6e0-d15bb44d4292 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:08.828641 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-79ba0985-caff-4c18-b6e0-d15bb44d4292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:08.886067 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba0985-caff-4c18-b6e0-d15bb44d4292 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:08.889568 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2da5c-7ec2-469b-a4d8-a25dfde4f43b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:08.890172 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2da5c-7ec2-469b-a4d8-a25dfde4f43b service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:08.890446 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2da5c-7ec2-469b-a4d8-a25dfde4f43b service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:08.890715 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2da5c-7ec2-469b-a4d8-a25dfde4f43b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:08.900392 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2da5c-7ec2-469b-a4d8-a25dfde4f43b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:08.901805 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2da5c-7ec2-469b-a4d8-a25dfde4f43b service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:08.910906 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1266/2521] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:33:09.550189 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17cc54-2c11-4b27-84a3-5c97e19ea4b4 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:09.550967 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17cc54-2c11-4b27-84a3-5c97e19ea4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:09.550967 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17cc54-2c11-4b27-84a3-5c97e19ea4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:09.586754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4a17cc54-2c11-4b27-84a3-5c97e19ea4b4 None None] MFA Rules not processed for user `73f3cd4c20404c99b53da6cb2bc33e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:09.588015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4a17cc54-2c11-4b27-84a3-5c97e19ea4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:09.656618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1256/2522] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:09.667638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2da5c-7ec2-469b-a4d8-a25dfde4f43b service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:09.668988 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9bb2da5c-7ec2-469b-a4d8-a25dfde4f43b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:09.729609 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2da5c-7ec2-469b-a4d8-a25dfde4f43b service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:09.733673 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6558406-9688-478a-83a8-7632de9b00ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:09.734294 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558406-9688-478a-83a8-7632de9b00ff service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:09.734572 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558406-9688-478a-83a8-7632de9b00ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:09.734842 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558406-9688-478a-83a8-7632de9b00ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:09.751382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6558406-9688-478a-83a8-7632de9b00ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:09.752858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6558406-9688-478a-83a8-7632de9b00ff service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:09.761833 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1267/2523] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:33:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:33:11.130708 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5778971-aebb-41ed-861e-f69d79d2a636 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:11.131242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5778971-aebb-41ed-861e-f69d79d2a636 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:11.131366 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5778971-aebb-41ed-861e-f69d79d2a636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:11.154656 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c5778971-aebb-41ed-861e-f69d79d2a636 None None] MFA Rules not processed for user `73f3cd4c20404c99b53da6cb2bc33e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:11.155925 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c5778971-aebb-41ed-861e-f69d79d2a636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:11.163876 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1257/2524] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:19.757827 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=85,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:21.144673 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=45 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:26.484506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc0d0d-186b-461b-beac-28ed93b3a678 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:26.484967 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc0d0d-186b-461b-beac-28ed93b3a678 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:26.485083 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc0d0d-186b-461b-beac-28ed93b3a678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:26.504936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f6dc0d0d-186b-461b-beac-28ed93b3a678 None None] MFA Rules not processed for user `73f3cd4c20404c99b53da6cb2bc33e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:26.506137 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f6dc0d0d-186b-461b-beac-28ed93b3a678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:26.512940 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1268/2525] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:36.498535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:38.352678 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5778971-aebb-41ed-861e-f69d79d2a636 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:38.354098 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c5778971-aebb-41ed-861e-f69d79d2a636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:38.367950 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5778971-aebb-41ed-861e-f69d79d2a636 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:38.372443 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4df98-1f97-4e15-ac77-4504de0813bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:38.373065 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4df98-1f97-4e15-ac77-4504de0813bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:38.373344 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4df98-1f97-4e15-ac77-4504de0813bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:38.373618 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4df98-1f97-4e15-ac77-4504de0813bc admin admin] PATH_INFO: `/v3/users/70df039b13b044969ffbf585850dac01` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:38.374712 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4df98-1f97-4e15-ac77-4504de0813bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=70df039b13b044969ffbf585850dac01)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:38.377358 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4df98-1f97-4e15-ac77-4504de0813bc admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:38.422215 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1eb4df98-1f97-4e15-ac77-4504de0813bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70df039b13b044969ffbf585850dac01'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:38.430279 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1eb4df98-1f97-4e15-ac77-4504de0813bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70df039b13b044969ffbf585850dac01'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:38.430885 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1eb4df98-1f97-4e15-ac77-4504de0813bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70df039b13b044969ffbf585850dac01'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:38.443696 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1eb4df98-1f97-4e15-ac77-4504de0813bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70df039b13b044969ffbf585850dac01'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:38.450784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1eb4df98-1f97-4e15-ac77-4504de0813bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70df039b13b044969ffbf585850dac01'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:38.456818 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1258/2526] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:33:38 2022] DELETE /identity/v3/users/70df039b13b044969ffbf585850dac01 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:38.461397 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6799a-5bc6-45e4-a7f3-a34d469d165a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:38.461893 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6799a-5bc6-45e4-a7f3-a34d469d165a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:38.462001 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6799a-5bc6-45e4-a7f3-a34d469d165a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:38.480849 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2bd6799a-5bc6-45e4-a7f3-a34d469d165a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:38.482033 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2bd6799a-5bc6-45e4-a7f3-a34d469d165a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:38.541969 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1269/2527] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:38.561027 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4df98-1f97-4e15-ac77-4504de0813bc admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:38.562449 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1eb4df98-1f97-4e15-ac77-4504de0813bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:38.619703 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4df98-1f97-4e15-ac77-4504de0813bc admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:38.623211 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8575d2-4eab-4ea4-adde-3a35c7141286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:38.623824 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8575d2-4eab-4ea4-adde-3a35c7141286 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:38.624097 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8575d2-4eab-4ea4-adde-3a35c7141286 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:38.624363 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8575d2-4eab-4ea4-adde-3a35c7141286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:38.634008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8575d2-4eab-4ea4-adde-3a35c7141286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:38.635458 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8575d2-4eab-4ea4-adde-3a35c7141286 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:38.644988 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1259/2528] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:33:39.070956 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6799a-5bc6-45e4-a7f3-a34d469d165a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:39.072880 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2bd6799a-5bc6-45e4-a7f3-a34d469d165a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:39.096777 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6799a-5bc6-45e4-a7f3-a34d469d165a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:39.102755 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d79cb1-af4b-4279-84b1-5b9bbf170283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:39.103128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d79cb1-af4b-4279-84b1-5b9bbf170283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:39.103889 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d79cb1-af4b-4279-84b1-5b9bbf170283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:39.104175 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d79cb1-af4b-4279-84b1-5b9bbf170283 admin admin] PATH_INFO: `/v3/projects/74c82e7c15074f45a70ca8260f4a67d5` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:39.106348 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d79cb1-af4b-4279-84b1-5b9bbf170283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74c82e7c15074f45a70ca8260f4a67d5)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:39.108324 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d79cb1-af4b-4279-84b1-5b9bbf170283 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:39.159103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69d79cb1-af4b-4279-84b1-5b9bbf170283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74c82e7c15074f45a70ca8260f4a67d5', 'request_id': 'req-69d79cb1-af4b-4279-84b1-5b9bbf170283'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:39.164749 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69d79cb1-af4b-4279-84b1-5b9bbf170283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74c82e7c15074f45a70ca8260f4a67d5', 'request_id': 'req-69d79cb1-af4b-4279-84b1-5b9bbf170283'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:39.171328 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69d79cb1-af4b-4279-84b1-5b9bbf170283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74c82e7c15074f45a70ca8260f4a67d5', 'request_id': 'req-69d79cb1-af4b-4279-84b1-5b9bbf170283'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:39.186883 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69d79cb1-af4b-4279-84b1-5b9bbf170283 admin admin] The token cache is being invalidate because project 74c82e7c15074f45a70ca8260f4a67d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:33:39.187128 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69d79cb1-af4b-4279-84b1-5b9bbf170283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:39.187873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-69d79cb1-af4b-4279-84b1-5b9bbf170283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:39.189873 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1270/2529] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:33:39 2022] DELETE /identity/v3/projects/74c82e7c15074f45a70ca8260f4a67d5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:39.195340 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4d23b-ecb3-4fe3-bfbe-a29e64e957b8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:39.195635 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4d23b-ecb3-4fe3-bfbe-a29e64e957b8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:39.195910 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4d23b-ecb3-4fe3-bfbe-a29e64e957b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:39.216639 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-2bf4d23b-ecb3-4fe3-bfbe-a29e64e957b8 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:39.217880 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2bf4d23b-ecb3-4fe3-bfbe-a29e64e957b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:39.290500 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1260/2530] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:39.348795 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429a055-2138-4565-a781-8cd64a484035 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:39.349149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429a055-2138-4565-a781-8cd64a484035 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:39.349430 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429a055-2138-4565-a781-8cd64a484035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:39.375312 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6429a055-2138-4565-a781-8cd64a484035 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:39.377670 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6429a055-2138-4565-a781-8cd64a484035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:39.386876 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1271/2531] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:39.391290 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4d23b-ecb3-4fe3-bfbe-a29e64e957b8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:39.401607 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4d23b-ecb3-4fe3-bfbe-a29e64e957b8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:39.406256 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96e875-35f5-4099-a817-32d8f3257d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:39.406946 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96e875-35f5-4099-a817-32d8f3257d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:39.407263 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96e875-35f5-4099-a817-32d8f3257d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:39.407709 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96e875-35f5-4099-a817-32d8f3257d83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:39.408408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96e875-35f5-4099-a817-32d8f3257d83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:33:39.408748 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96e875-35f5-4099-a817-32d8f3257d83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:39.410452 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96e875-35f5-4099-a817-32d8f3257d83 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:39.423637 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1261/2532] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:33:39.428229 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429a055-2138-4565-a781-8cd64a484035 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:39.437537 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429a055-2138-4565-a781-8cd64a484035 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:39.442884 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71af179-9bfc-40a3-b5e3-fb9b97ae0f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:39.443510 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71af179-9bfc-40a3-b5e3-fb9b97ae0f50 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:39.443788 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71af179-9bfc-40a3-b5e3-fb9b97ae0f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:39.444055 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71af179-9bfc-40a3-b5e3-fb9b97ae0f50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:39.444684 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71af179-9bfc-40a3-b5e3-fb9b97ae0f50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:39.446559 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71af179-9bfc-40a3-b5e3-fb9b97ae0f50 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:39.473778 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1272/2533] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:33:39 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:33:39.477962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96e875-35f5-4099-a817-32d8f3257d83 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:39.484195 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:39.490997 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96e875-35f5-4099-a817-32d8f3257d83 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:39.553560 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba98c5b-1242-4724-b358-551886e180f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:39.554212 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba98c5b-1242-4724-b358-551886e180f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:39.554494 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba98c5b-1242-4724-b358-551886e180f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:39.554768 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba98c5b-1242-4724-b358-551886e180f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:39.555430 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba98c5b-1242-4724-b358-551886e180f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:39.557341 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba98c5b-1242-4724-b358-551886e180f7 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:39.578502 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1262/2534] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:33:39 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:33:39.582644 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71af179-9bfc-40a3-b5e3-fb9b97ae0f50 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:39.593088 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71af179-9bfc-40a3-b5e3-fb9b97ae0f50 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:39.598356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744987e-2565-4640-9fe8-b27d7c011d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:39.598955 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744987e-2565-4640-9fe8-b27d7c011d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:39.599395 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744987e-2565-4640-9fe8-b27d7c011d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:39.599672 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744987e-2565-4640-9fe8-b27d7c011d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:39.600350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744987e-2565-4640-9fe8-b27d7c011d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:39.601947 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744987e-2565-4640-9fe8-b27d7c011d4b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:39.613273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1273/2535] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:33:39.617205 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba98c5b-1242-4724-b358-551886e180f7 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:39.628666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba98c5b-1242-4724-b358-551886e180f7 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:39.633323 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a669ea3-5b28-46be-a35c-62ef3b27699b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:39.633973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669ea3-5b28-46be-a35c-62ef3b27699b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:39.634253 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669ea3-5b28-46be-a35c-62ef3b27699b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:39.634525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669ea3-5b28-46be-a35c-62ef3b27699b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:39.635229 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a669ea3-5b28-46be-a35c-62ef3b27699b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:39.636818 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a669ea3-5b28-46be-a35c-62ef3b27699b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:39.648104 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1263/2536] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:33:39.652021 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744987e-2565-4640-9fe8-b27d7c011d4b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:39.666580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744987e-2565-4640-9fe8-b27d7c011d4b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:39.671419 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f4136-0e52-4e66-a28a-f3a61c8f72ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:39.672075 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f4136-0e52-4e66-a28a-f3a61c8f72ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:39.672354 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f4136-0e52-4e66-a28a-f3a61c8f72ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:39.672623 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f4136-0e52-4e66-a28a-f3a61c8f72ab admin admin] PATH_INFO: `/v3/projects/fa1039ddd2c3414aa481e9a9ccf476a4/users/1f999711e7d644f5a8ae2cf1b2c3f8a4/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:39.688323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f4136-0e52-4e66-a28a-f3a61c8f72ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1039ddd2c3414aa481e9a9ccf476a4, user_id=1f999711e7d644f5a8ae2cf1b2c3f8a4, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:39.691412 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f4136-0e52-4e66-a28a-f3a61c8f72ab admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:39.704103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1274/2537] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:33:39 2022] PUT /identity/v3/projects/fa1039ddd2c3414aa481e9a9ccf476a4/users/1f999711e7d644f5a8ae2cf1b2c3f8a4/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:39.714016 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025abcfa-ad5b-4161-a3e6-678bcbcc4eb4 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:39.714298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025abcfa-ad5b-4161-a3e6-678bcbcc4eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:39.714573 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025abcfa-ad5b-4161-a3e6-678bcbcc4eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:39.745998 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-025abcfa-ad5b-4161-a3e6-678bcbcc4eb4 None None] MFA Rules not processed for user `1f999711e7d644f5a8ae2cf1b2c3f8a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:39.747240 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-025abcfa-ad5b-4161-a3e6-678bcbcc4eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:39.816715 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1264/2538] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:39.828020 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f4136-0e52-4e66-a28a-f3a61c8f72ab admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:39.830520 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-913f4136-0e52-4e66-a28a-f3a61c8f72ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:39.854716 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:39.897530 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f4136-0e52-4e66-a28a-f3a61c8f72ab admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:39.901187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07745a-8a23-416d-89de-f579b66f83c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:39.901798 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07745a-8a23-416d-89de-f579b66f83c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:39.902080 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07745a-8a23-416d-89de-f579b66f83c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:39.902350 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07745a-8a23-416d-89de-f579b66f83c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:39.911533 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07745a-8a23-416d-89de-f579b66f83c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:39.912973 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07745a-8a23-416d-89de-f579b66f83c8 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:39.921785 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1275/2539] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:33:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:33:44.199249 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd9e28-ab50-4b61-97ed-503bf6ad8dc2 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:44.199744 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd9e28-ab50-4b61-97ed-503bf6ad8dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:44.199851 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd9e28-ab50-4b61-97ed-503bf6ad8dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:44.232528 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f3dd9e28-ab50-4b61-97ed-503bf6ad8dc2 None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:44.233736 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f3dd9e28-ab50-4b61-97ed-503bf6ad8dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:44.287584 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1265/2540] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:33:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:44.334244 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07745a-8a23-416d-89de-f579b66f83c8 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:44.335730 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ac07745a-8a23-416d-89de-f579b66f83c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:44.404049 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07745a-8a23-416d-89de-f579b66f83c8 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:44.409095 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72513a9a-681a-4b72-a18d-866942220389 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:44.409696 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72513a9a-681a-4b72-a18d-866942220389 service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:44.409972 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72513a9a-681a-4b72-a18d-866942220389 service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:44.410243 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72513a9a-681a-4b72-a18d-866942220389 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:44.418188 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72513a9a-681a-4b72-a18d-866942220389 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:44.419787 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72513a9a-681a-4b72-a18d-866942220389 service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:44.428230 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1276/2541] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:33:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:33:44.616079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd9e28-ab50-4b61-97ed-503bf6ad8dc2 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:44.628796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:44.692491 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd9e28-ab50-4b61-97ed-503bf6ad8dc2 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:44.697686 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d41ab-6de2-4429-9ed6-acab6a2084dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:44.698605 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d41ab-6de2-4429-9ed6-acab6a2084dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:44.699113 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d41ab-6de2-4429-9ed6-acab6a2084dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:44.699614 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d41ab-6de2-4429-9ed6-acab6a2084dc admin admin] PATH_INFO: `/v3/users/1f999711e7d644f5a8ae2cf1b2c3f8a4` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:44.701233 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d41ab-6de2-4429-9ed6-acab6a2084dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f999711e7d644f5a8ae2cf1b2c3f8a4)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:44.703183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d41ab-6de2-4429-9ed6-acab6a2084dc admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:44.764893 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-064d41ab-6de2-4429-9ed6-acab6a2084dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f999711e7d644f5a8ae2cf1b2c3f8a4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:44.773224 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-064d41ab-6de2-4429-9ed6-acab6a2084dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f999711e7d644f5a8ae2cf1b2c3f8a4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:44.774079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-064d41ab-6de2-4429-9ed6-acab6a2084dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f999711e7d644f5a8ae2cf1b2c3f8a4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:44.788811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-064d41ab-6de2-4429-9ed6-acab6a2084dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f999711e7d644f5a8ae2cf1b2c3f8a4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:44.795834 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-064d41ab-6de2-4429-9ed6-acab6a2084dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f999711e7d644f5a8ae2cf1b2c3f8a4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:44.803333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1266/2542] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:33:44 2022] DELETE /identity/v3/users/1f999711e7d644f5a8ae2cf1b2c3f8a4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:33:44.807991 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e12cbc-6b04-4dd1-a468-329a68118a5c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:44.808473 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e12cbc-6b04-4dd1-a468-329a68118a5c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:44.808843 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e12cbc-6b04-4dd1-a468-329a68118a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:44.834156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a8e12cbc-6b04-4dd1-a468-329a68118a5c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:44.835384 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a8e12cbc-6b04-4dd1-a468-329a68118a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:44.924209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1277/2543] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:44.933762 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d41ab-6de2-4429-9ed6-acab6a2084dc admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:44.935275 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-064d41ab-6de2-4429-9ed6-acab6a2084dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:45.002511 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d41ab-6de2-4429-9ed6-acab6a2084dc admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:45.006806 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ed3f3-9f20-458b-a310-19d492f68769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:45.007568 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ed3f3-9f20-458b-a310-19d492f68769 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:45.007848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ed3f3-9f20-458b-a310-19d492f68769 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:45.008120 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ed3f3-9f20-458b-a310-19d492f68769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:45.019030 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ed3f3-9f20-458b-a310-19d492f68769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:45.021163 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ed3f3-9f20-458b-a310-19d492f68769 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:45.033094 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1267/2544] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:33:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:33:45.418083 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e12cbc-6b04-4dd1-a468-329a68118a5c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:33:45.421323 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a8e12cbc-6b04-4dd1-a468-329a68118a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:45.448907 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e12cbc-6b04-4dd1-a468-329a68118a5c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:33:45.455270 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:33:45.456063 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:45.456342 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:45.456615 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56 admin admin] PATH_INFO: `/v3/projects/fa1039ddd2c3414aa481e9a9ccf476a4` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:45.457752 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa1039ddd2c3414aa481e9a9ccf476a4)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:33:45.459751 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:33:45.518377 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:45.521151 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa1039ddd2c3414aa481e9a9ccf476a4', 'request_id': 'req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:45.528831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa1039ddd2c3414aa481e9a9ccf476a4', 'request_id': 'req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:45.536866 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa1039ddd2c3414aa481e9a9ccf476a4', 'request_id': 'req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:45.553361 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56 admin admin] The token cache is being invalidate because project fa1039ddd2c3414aa481e9a9ccf476a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:33:45.553702 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:45.554278 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d6c5d875-a2ec-4ee5-aa21-9e58c076bb56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:33:45.556085 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1278/2545] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:33:45 2022] DELETE /identity/v3/projects/fa1039ddd2c3414aa481e9a9ccf476a4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:33:52.807598 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb24f10-aa6b-4e42-8dec-2e8b3753e31d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:33:52.808089 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb24f10-aa6b-4e42-8dec-2e8b3753e31d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:33:52.808189 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb24f10-aa6b-4e42-8dec-2e8b3753e31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:33:52.833838 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9bb24f10-aa6b-4e42-8dec-2e8b3753e31d None None] MFA Rules not processed for user `eca98759b55b443b9591eb6c80501157`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:33:52.835106 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9bb24f10-aa6b-4e42-8dec-2e8b3753e31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:33:52.863408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:33:52.905133 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1268/2546] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:33:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:33:55.550262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=7,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:34:00.388909 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8825f14-9577-49f5-a949-2a0712d2b912 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:00.389463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8825f14-9577-49f5-a949-2a0712d2b912 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:00.389463 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8825f14-9577-49f5-a949-2a0712d2b912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:00.410826 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e8825f14-9577-49f5-a949-2a0712d2b912 None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:34:00.412107 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e8825f14-9577-49f5-a949-2a0712d2b912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:00.465552 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1279/2547] 10.222.0.18 () {58 vars in 967 bytes} [Fri Oct 14 02:34:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:34:00.478298 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb24f10-aa6b-4e42-8dec-2e8b3753e31d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:00.479971 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9bb24f10-aa6b-4e42-8dec-2e8b3753e31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:00.548962 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb24f10-aa6b-4e42-8dec-2e8b3753e31d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:00.553857 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4508079-39e1-443b-a0af-a71f52ed401f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:00.554478 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4508079-39e1-443b-a0af-a71f52ed401f service swift] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:00.554758 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4508079-39e1-443b-a0af-a71f52ed401f service swift] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:00.555034 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4508079-39e1-443b-a0af-a71f52ed401f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:00.563548 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4508079-39e1-443b-a0af-a71f52ed401f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:00.565131 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4508079-39e1-443b-a0af-a71f52ed401f service swift] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:00.573026 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1269/2548] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:34:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:34:02.465999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8825f14-9577-49f5-a949-2a0712d2b912 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:02.467597 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e8825f14-9577-49f5-a949-2a0712d2b912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:02.540421 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8825f14-9577-49f5-a949-2a0712d2b912 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:02.544936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d2e2d-2e84-4d9d-9c7b-22d2114bc874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:02.545567 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d2e2d-2e84-4d9d-9c7b-22d2114bc874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:02.545843 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d2e2d-2e84-4d9d-9c7b-22d2114bc874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:02.546114 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d2e2d-2e84-4d9d-9c7b-22d2114bc874 admin admin] PATH_INFO: `/v3/users/73f3cd4c20404c99b53da6cb2bc33e84` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:02.547914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d2e2d-2e84-4d9d-9c7b-22d2114bc874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73f3cd4c20404c99b53da6cb2bc33e84)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:02.549818 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d2e2d-2e84-4d9d-9c7b-22d2114bc874 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:02.594084 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8a0d2e2d-2e84-4d9d-9c7b-22d2114bc874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73f3cd4c20404c99b53da6cb2bc33e84'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:02.602541 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8a0d2e2d-2e84-4d9d-9c7b-22d2114bc874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73f3cd4c20404c99b53da6cb2bc33e84'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:02.614920 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8a0d2e2d-2e84-4d9d-9c7b-22d2114bc874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73f3cd4c20404c99b53da6cb2bc33e84'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:02.621542 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8a0d2e2d-2e84-4d9d-9c7b-22d2114bc874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73f3cd4c20404c99b53da6cb2bc33e84'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:02.622172 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8a0d2e2d-2e84-4d9d-9c7b-22d2114bc874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73f3cd4c20404c99b53da6cb2bc33e84'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:02.628935 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1280/2549] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:34:02 2022] DELETE /identity/v3/users/73f3cd4c20404c99b53da6cb2bc33e84 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:34:02.793991 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4508079-39e1-443b-a0af-a71f52ed401f service swift] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:02.806872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4508079-39e1-443b-a0af-a71f52ed401f service swift] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:02.864553 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb309d-d773-453f-ae2b-23d3c01d92d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:02.865190 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb309d-d773-453f-ae2b-23d3c01d92d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:02.865474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb309d-d773-453f-ae2b-23d3c01d92d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:02.865752 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb309d-d773-453f-ae2b-23d3c01d92d0 admin admin] PATH_INFO: `/v3/projects/d50887bbbe1d4a66bdae02e0360129fb` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:02.866870 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb309d-d773-453f-ae2b-23d3c01d92d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d50887bbbe1d4a66bdae02e0360129fb)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:02.868810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb309d-d773-453f-ae2b-23d3c01d92d0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:02.913721 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9aeb309d-d773-453f-ae2b-23d3c01d92d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd50887bbbe1d4a66bdae02e0360129fb', 'request_id': 'req-9aeb309d-d773-453f-ae2b-23d3c01d92d0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:02.919102 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9aeb309d-d773-453f-ae2b-23d3c01d92d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd50887bbbe1d4a66bdae02e0360129fb', 'request_id': 'req-9aeb309d-d773-453f-ae2b-23d3c01d92d0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:02.925667 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9aeb309d-d773-453f-ae2b-23d3c01d92d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd50887bbbe1d4a66bdae02e0360129fb', 'request_id': 'req-9aeb309d-d773-453f-ae2b-23d3c01d92d0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:02.928178 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:34:02.939704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9aeb309d-d773-453f-ae2b-23d3c01d92d0 admin admin] The token cache is being invalidate because project d50887bbbe1d4a66bdae02e0360129fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:34:02.939992 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9aeb309d-d773-453f-ae2b-23d3c01d92d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:02.940651 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9aeb309d-d773-453f-ae2b-23d3c01d92d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:02.942155 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1270/2550] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:34:02 2022] DELETE /identity/v3/projects/d50887bbbe1d4a66bdae02e0360129fb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:34:02.947011 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dd7b2-2318-4c33-90fd-9e242865596c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:02.947330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dd7b2-2318-4c33-90fd-9e242865596c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:02.947631 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dd7b2-2318-4c33-90fd-9e242865596c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:02.966906 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a38dd7b2-2318-4c33-90fd-9e242865596c None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:34:02.968140 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a38dd7b2-2318-4c33-90fd-9e242865596c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:02.987865 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:34:03.029510 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1281/2551] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:34:03.092073 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f8646-d484-4e26-a64c-9298ad59af98 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:03.092291 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f8646-d484-4e26-a64c-9298ad59af98 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:03.092532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f8646-d484-4e26-a64c-9298ad59af98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:03.113559 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-901f8646-d484-4e26-a64c-9298ad59af98 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:34:03.114773 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-901f8646-d484-4e26-a64c-9298ad59af98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:03.122694 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1271/2552] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:34:03.127171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dd7b2-2318-4c33-90fd-9e242865596c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:03.137010 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dd7b2-2318-4c33-90fd-9e242865596c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:03.142069 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bb541-eb3c-46bf-b26d-8b9aba5d1815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:03.142669 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bb541-eb3c-46bf-b26d-8b9aba5d1815 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:03.142948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bb541-eb3c-46bf-b26d-8b9aba5d1815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:03.143262 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bb541-eb3c-46bf-b26d-8b9aba5d1815 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:03.143914 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68bb541-eb3c-46bf-b26d-8b9aba5d1815 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:34:03.144216 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68bb541-eb3c-46bf-b26d-8b9aba5d1815 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:03.145813 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68bb541-eb3c-46bf-b26d-8b9aba5d1815 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:03.157729 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1282/2553] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:34:03.161879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f8646-d484-4e26-a64c-9298ad59af98 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:03.170916 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f8646-d484-4e26-a64c-9298ad59af98 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:03.175822 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592eca5f-bcba-437e-9caa-abff2d28874c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:03.176419 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592eca5f-bcba-437e-9caa-abff2d28874c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:03.176699 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592eca5f-bcba-437e-9caa-abff2d28874c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:03.176973 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592eca5f-bcba-437e-9caa-abff2d28874c admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:03.177602 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592eca5f-bcba-437e-9caa-abff2d28874c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:03.179532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592eca5f-bcba-437e-9caa-abff2d28874c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:03.206642 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1272/2554] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:34:03 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:34:03.210769 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bb541-eb3c-46bf-b26d-8b9aba5d1815 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:03.221401 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bb541-eb3c-46bf-b26d-8b9aba5d1815 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:03.276330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28306410-a30b-4048-bd5f-8cc14f8a1e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:03.276934 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28306410-a30b-4048-bd5f-8cc14f8a1e62 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:03.277218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28306410-a30b-4048-bd5f-8cc14f8a1e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:03.277495 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28306410-a30b-4048-bd5f-8cc14f8a1e62 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:03.278124 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28306410-a30b-4048-bd5f-8cc14f8a1e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:03.280219 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28306410-a30b-4048-bd5f-8cc14f8a1e62 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:03.301516 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1283/2555] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:34:03 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:34:03.305805 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592eca5f-bcba-437e-9caa-abff2d28874c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:03.315936 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592eca5f-bcba-437e-9caa-abff2d28874c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:03.320520 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaa948-5f97-48f4-a3ba-6781aa2c100f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:03.321117 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaa948-5f97-48f4-a3ba-6781aa2c100f admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:03.321395 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaa948-5f97-48f4-a3ba-6781aa2c100f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:03.321664 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaa948-5f97-48f4-a3ba-6781aa2c100f admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:03.322295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaa948-5f97-48f4-a3ba-6781aa2c100f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:03.323900 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaa948-5f97-48f4-a3ba-6781aa2c100f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:03.334499 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1273/2556] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:34:03.338298 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28306410-a30b-4048-bd5f-8cc14f8a1e62 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:03.349035 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28306410-a30b-4048-bd5f-8cc14f8a1e62 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:03.353566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27511ed5-7116-49cf-bd82-95316617b3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:03.354427 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27511ed5-7116-49cf-bd82-95316617b3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:03.354707 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27511ed5-7116-49cf-bd82-95316617b3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:03.354978 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27511ed5-7116-49cf-bd82-95316617b3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:03.355647 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27511ed5-7116-49cf-bd82-95316617b3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:03.357249 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27511ed5-7116-49cf-bd82-95316617b3e7 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:03.367692 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1284/2557] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:34:03.371668 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaa948-5f97-48f4-a3ba-6781aa2c100f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:03.381844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaa948-5f97-48f4-a3ba-6781aa2c100f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:03.386735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35552ed8-967a-41df-810b-565afba16c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:03.387408 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35552ed8-967a-41df-810b-565afba16c3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:03.387688 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35552ed8-967a-41df-810b-565afba16c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:03.387963 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35552ed8-967a-41df-810b-565afba16c3a admin admin] PATH_INFO: `/v3/projects/f766b1819251495a93c7cc12d672d4c3/users/78b895ad16ac44d4b7eb1af2c62078d6/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:03.404232 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35552ed8-967a-41df-810b-565afba16c3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f766b1819251495a93c7cc12d672d4c3, user_id=78b895ad16ac44d4b7eb1af2c62078d6, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:03.408049 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35552ed8-967a-41df-810b-565afba16c3a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:03.420679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1274/2558] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:34:03 2022] PUT /identity/v3/projects/f766b1819251495a93c7cc12d672d4c3/users/78b895ad16ac44d4b7eb1af2c62078d6/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:34:03.431091 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebf719-4a2c-4f00-b9cc-d68aacd7ee10 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:03.431382 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebf719-4a2c-4f00-b9cc-d68aacd7ee10 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:03.431652 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebf719-4a2c-4f00-b9cc-d68aacd7ee10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:03.459168 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e4ebf719-4a2c-4f00-b9cc-d68aacd7ee10 None None] MFA Rules not processed for user `78b895ad16ac44d4b7eb1af2c62078d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:34:03.460356 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e4ebf719-4a2c-4f00-b9cc-d68aacd7ee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:03.523184 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1285/2559] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:34:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:34:03.534409 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35552ed8-967a-41df-810b-565afba16c3a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:03.536226 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-35552ed8-967a-41df-810b-565afba16c3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:03.596672 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35552ed8-967a-41df-810b-565afba16c3a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:03.600562 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c3043-ed57-4d3c-846c-d234d183a35c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:03.601110 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c3043-ed57-4d3c-846c-d234d183a35c service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:03.601390 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c3043-ed57-4d3c-846c-d234d183a35c service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:03.601661 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c3043-ed57-4d3c-846c-d234d183a35c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:03.610527 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c3043-ed57-4d3c-846c-d234d183a35c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:03.611984 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c3043-ed57-4d3c-846c-d234d183a35c service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:03.620889 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1275/2560] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:34:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:34:13.519766 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:34:13.616848 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=1,keystone:SELECT=55,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:34:37.686030 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebf719-4a2c-4f00-b9cc-d68aacd7ee10 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:37.753958 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebf719-4a2c-4f00-b9cc-d68aacd7ee10 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:37.758398 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c170c-8cbe-4066-9465-905937030b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:37.759014 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c170c-8cbe-4066-9465-905937030b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:37.759333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c170c-8cbe-4066-9465-905937030b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:37.759607 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c170c-8cbe-4066-9465-905937030b87 admin admin] PATH_INFO: `/v3/users/78b895ad16ac44d4b7eb1af2c62078d6` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:37.760726 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c170c-8cbe-4066-9465-905937030b87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b895ad16ac44d4b7eb1af2c62078d6)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:37.762686 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c170c-8cbe-4066-9465-905937030b87 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:37.817046 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-622c170c-8cbe-4066-9465-905937030b87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b895ad16ac44d4b7eb1af2c62078d6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:37.823802 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-622c170c-8cbe-4066-9465-905937030b87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b895ad16ac44d4b7eb1af2c62078d6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:37.836408 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-622c170c-8cbe-4066-9465-905937030b87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b895ad16ac44d4b7eb1af2c62078d6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:37.843218 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-622c170c-8cbe-4066-9465-905937030b87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b895ad16ac44d4b7eb1af2c62078d6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:37.843856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-622c170c-8cbe-4066-9465-905937030b87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b895ad16ac44d4b7eb1af2c62078d6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:37.850360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1286/2561] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:34:37 2022] DELETE /identity/v3/users/78b895ad16ac44d4b7eb1af2c62078d6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:34:37.855285 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3bc08-19bc-424d-b3b2-805e04b8393e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:37.855766 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3bc08-19bc-424d-b3b2-805e04b8393e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:37.855872 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3bc08-19bc-424d-b3b2-805e04b8393e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:37.874929 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-2cc3bc08-19bc-424d-b3b2-805e04b8393e None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:34:37.876722 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2cc3bc08-19bc-424d-b3b2-805e04b8393e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:37.934356 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1276/2562] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:34:37.944517 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c170c-8cbe-4066-9465-905937030b87 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:37.945907 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-622c170c-8cbe-4066-9465-905937030b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:38.005584 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c170c-8cbe-4066-9465-905937030b87 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:38.009089 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1c594-e8d1-4e28-8ab9-22fc8381cadf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:38.009723 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c594-e8d1-4e28-8ab9-22fc8381cadf service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:38.010027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c594-e8d1-4e28-8ab9-22fc8381cadf service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:38.010296 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c594-e8d1-4e28-8ab9-22fc8381cadf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:38.019953 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1c594-e8d1-4e28-8ab9-22fc8381cadf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:38.021405 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1c594-e8d1-4e28-8ab9-22fc8381cadf service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:38.030718 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1287/2563] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:34:38.548081 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3bc08-19bc-424d-b3b2-805e04b8393e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:38.549715 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2cc3bc08-19bc-424d-b3b2-805e04b8393e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:38.562054 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3bc08-19bc-424d-b3b2-805e04b8393e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:38.569595 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ef017-3cf2-45dc-b3cd-7c8432580c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:38.570669 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ef017-3cf2-45dc-b3cd-7c8432580c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:38.571197 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ef017-3cf2-45dc-b3cd-7c8432580c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:38.571525 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ef017-3cf2-45dc-b3cd-7c8432580c44 admin admin] PATH_INFO: `/v3/projects/f766b1819251495a93c7cc12d672d4c3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:38.573339 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ef017-3cf2-45dc-b3cd-7c8432580c44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f766b1819251495a93c7cc12d672d4c3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:38.575532 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ef017-3cf2-45dc-b3cd-7c8432580c44 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:38.638022 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-537ef017-3cf2-45dc-b3cd-7c8432580c44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f766b1819251495a93c7cc12d672d4c3', 'request_id': 'req-537ef017-3cf2-45dc-b3cd-7c8432580c44'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:38.644210 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-537ef017-3cf2-45dc-b3cd-7c8432580c44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f766b1819251495a93c7cc12d672d4c3', 'request_id': 'req-537ef017-3cf2-45dc-b3cd-7c8432580c44'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:38.650237 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-537ef017-3cf2-45dc-b3cd-7c8432580c44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f766b1819251495a93c7cc12d672d4c3', 'request_id': 'req-537ef017-3cf2-45dc-b3cd-7c8432580c44'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:38.662779 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-537ef017-3cf2-45dc-b3cd-7c8432580c44 admin admin] The token cache is being invalidate because project f766b1819251495a93c7cc12d672d4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:34:38.663115 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-537ef017-3cf2-45dc-b3cd-7c8432580c44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:38.663726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-537ef017-3cf2-45dc-b3cd-7c8432580c44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:38.665213 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1277/2564] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:34:38 2022] DELETE /identity/v3/projects/f766b1819251495a93c7cc12d672d4c3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:34:38.670078 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940eadc-2019-453e-8a8e-0cfb3fd4099d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:38.670365 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940eadc-2019-453e-8a8e-0cfb3fd4099d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:38.670646 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940eadc-2019-453e-8a8e-0cfb3fd4099d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:38.690343 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3940eadc-2019-453e-8a8e-0cfb3fd4099d None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:34:38.691574 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3940eadc-2019-453e-8a8e-0cfb3fd4099d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:38.709336 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:34:38.765149 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1288/2565] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:34:38.775740 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde327e0-faad-4601-b381-62605e376b4a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:38.776015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde327e0-faad-4601-b381-62605e376b4a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:38.776283 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde327e0-faad-4601-b381-62605e376b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:38.797037 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-fde327e0-faad-4601-b381-62605e376b4a None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:34:38.798599 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-fde327e0-faad-4601-b381-62605e376b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:38.807454 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1278/2566] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:34:38.812187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940eadc-2019-453e-8a8e-0cfb3fd4099d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:38.826387 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940eadc-2019-453e-8a8e-0cfb3fd4099d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:38.834640 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad062c3-aea1-4993-b672-674a6666605b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:38.835285 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad062c3-aea1-4993-b672-674a6666605b admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:38.835566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad062c3-aea1-4993-b672-674a6666605b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:38.835830 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad062c3-aea1-4993-b672-674a6666605b admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:38.836687 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad062c3-aea1-4993-b672-674a6666605b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:34:38.837123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad062c3-aea1-4993-b672-674a6666605b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:38.838811 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad062c3-aea1-4993-b672-674a6666605b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:38.853224 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1289/2567] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Oct 14 02:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:34:38.857696 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde327e0-faad-4601-b381-62605e376b4a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:38.867853 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde327e0-faad-4601-b381-62605e376b4a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:38.872495 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6e4cc-8ae4-48a2-b395-e05fcdf6d2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:38.873417 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6e4cc-8ae4-48a2-b395-e05fcdf6d2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:38.873698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6e4cc-8ae4-48a2-b395-e05fcdf6d2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:38.873966 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6e4cc-8ae4-48a2-b395-e05fcdf6d2f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:38.874601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d6e4cc-8ae4-48a2-b395-e05fcdf6d2f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:38.877127 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d6e4cc-8ae4-48a2-b395-e05fcdf6d2f1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:38.919811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1279/2568] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Oct 14 02:34:38 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:34:38.924126 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad062c3-aea1-4993-b672-674a6666605b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:38.934286 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad062c3-aea1-4993-b672-674a6666605b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:38.993745 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edd66c-7749-40eb-b342-a39ff7c3c735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:38.994359 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edd66c-7749-40eb-b342-a39ff7c3c735 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:38.994640 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edd66c-7749-40eb-b342-a39ff7c3c735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:38.994921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edd66c-7749-40eb-b342-a39ff7c3c735 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:38.995612 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1edd66c-7749-40eb-b342-a39ff7c3c735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:38.997558 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1edd66c-7749-40eb-b342-a39ff7c3c735 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:39.021594 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1290/2569] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Oct 14 02:34:38 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:34:39.025474 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6e4cc-8ae4-48a2-b395-e05fcdf6d2f1 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:39.035698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6e4cc-8ae4-48a2-b395-e05fcdf6d2f1 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:39.040893 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75adaa-293e-4721-8279-d2cfd7dd2aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:39.041518 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75adaa-293e-4721-8279-d2cfd7dd2aec admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:39.041793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75adaa-293e-4721-8279-d2cfd7dd2aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:39.042058 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75adaa-293e-4721-8279-d2cfd7dd2aec admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:39.042756 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75adaa-293e-4721-8279-d2cfd7dd2aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:39.044544 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75adaa-293e-4721-8279-d2cfd7dd2aec admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:39.057624 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1280/2570] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:34:39.061688 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edd66c-7749-40eb-b342-a39ff7c3c735 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:39.072634 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edd66c-7749-40eb-b342-a39ff7c3c735 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:39.077468 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c162b4f-cf9f-4d05-a6ea-1350f0b59dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:39.078077 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c162b4f-cf9f-4d05-a6ea-1350f0b59dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:39.078361 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c162b4f-cf9f-4d05-a6ea-1350f0b59dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:39.078638 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c162b4f-cf9f-4d05-a6ea-1350f0b59dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:39.079430 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c162b4f-cf9f-4d05-a6ea-1350f0b59dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:39.081178 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c162b4f-cf9f-4d05-a6ea-1350f0b59dd4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:39.092898 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1291/2571] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Oct 14 02:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:34:39.097850 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75adaa-293e-4721-8279-d2cfd7dd2aec admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:39.109666 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75adaa-293e-4721-8279-d2cfd7dd2aec admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:39.115704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fc787-93ea-46da-b000-2178e5348ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:39.116370 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fc787-93ea-46da-b000-2178e5348ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:39.116654 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fc787-93ea-46da-b000-2178e5348ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:39.116932 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fc787-93ea-46da-b000-2178e5348ae2 admin admin] PATH_INFO: `/v3/projects/49bdcf3d2b67466badfe669519fe7f44/users/522a06b7de354628820de2e35ae40b63/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:39.133589 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fc787-93ea-46da-b000-2178e5348ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49bdcf3d2b67466badfe669519fe7f44, user_id=522a06b7de354628820de2e35ae40b63, role_id=e460c7cf7b5b4c35a5a0e326f69a9fe3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:39.136312 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fc787-93ea-46da-b000-2178e5348ae2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:39.151546 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1281/2572] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Oct 14 02:34:39 2022] PUT /identity/v3/projects/49bdcf3d2b67466badfe669519fe7f44/users/522a06b7de354628820de2e35ae40b63/roles/e460c7cf7b5b4c35a5a0e326f69a9fe3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:34:39.161903 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef1835-5b95-4894-8c4f-d04ee5325374 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:39.162181 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef1835-5b95-4894-8c4f-d04ee5325374 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:39.162566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef1835-5b95-4894-8c4f-d04ee5325374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:39.194119 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e3ef1835-5b95-4894-8c4f-d04ee5325374 None None] MFA Rules not processed for user `522a06b7de354628820de2e35ae40b63`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:34:39.195388 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e3ef1835-5b95-4894-8c4f-d04ee5325374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:39.270269 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1292/2573] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:34:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:34:39.282775 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fc787-93ea-46da-b000-2178e5348ae2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:39.284754 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-754fc787-93ea-46da-b000-2178e5348ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:39.300270 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:34:39.346750 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fc787-93ea-46da-b000-2178e5348ae2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:39.350425 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc1471-c8d0-4089-bacf-df5b75c88fda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7a43e284e1401688b2e202c67a5eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:39.351038 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc1471-c8d0-4089-bacf-df5b75c88fda service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:39.351350 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc1471-c8d0-4089-bacf-df5b75c88fda service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:39.351622 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc1471-c8d0-4089-bacf-df5b75c88fda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:39.360254 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc1471-c8d0-4089-bacf-df5b75c88fda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:39.362241 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc1471-c8d0-4089-bacf-df5b75c88fda service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:39.370879 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1282/2574] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:34:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:34:40.724457 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a61d2-a47a-4549-87bc-c8d8937a2907 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:40.724999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a61d2-a47a-4549-87bc-c8d8937a2907 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:40.724999 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a61d2-a47a-4549-87bc-c8d8937a2907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:40.744682 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-570a61d2-a47a-4549-87bc-c8d8937a2907 None None] MFA Rules not processed for user `522a06b7de354628820de2e35ae40b63`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:34:40.745858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-570a61d2-a47a-4549-87bc-c8d8937a2907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:40.753326 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1293/2575] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:34:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:34:40.789595 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc1471-c8d0-4089-bacf-df5b75c88fda service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:40.791002 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-64dc1471-c8d0-4089-bacf-df5b75c88fda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:40.846793 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc1471-c8d0-4089-bacf-df5b75c88fda service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:40.850415 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ed766-510c-4631-bd30-e625f8a36b55 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c05a1b303c2418fb2941b8ef56f8972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:40.851018 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ed766-510c-4631-bd30-e625f8a36b55 service glance] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:40.851339 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ed766-510c-4631-bd30-e625f8a36b55 service glance] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:40.851613 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ed766-510c-4631-bd30-e625f8a36b55 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:40.853572 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-335ed766-510c-4631-bd30-e625f8a36b55 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:40.923391 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ed766-510c-4631-bd30-e625f8a36b55 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:40.924711 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ed766-510c-4631-bd30-e625f8a36b55 service glance] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:40.935504 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1283/2576] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:34:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:34:41.373148 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a61d2-a47a-4549-87bc-c8d8937a2907 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:41.374580 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-570a61d2-a47a-4549-87bc-c8d8937a2907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:41.387373 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:34:41.434728 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a61d2-a47a-4549-87bc-c8d8937a2907 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:41.438779 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44aa76-b374-4ee6-b8e3-cc2831edbff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:41.439425 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44aa76-b374-4ee6-b8e3-cc2831edbff8 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:41.439707 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44aa76-b374-4ee6-b8e3-cc2831edbff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:41.439982 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44aa76-b374-4ee6-b8e3-cc2831edbff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:41.441733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8d44aa76-b374-4ee6-b8e3-cc2831edbff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:41.456065 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44aa76-b374-4ee6-b8e3-cc2831edbff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:41.457617 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44aa76-b374-4ee6-b8e3-cc2831edbff8 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:41.468666 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1294/2577] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:34:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:34:50.930634 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=75 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:34:51.463598 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=21 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:34:53.651860 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165a371-6aa2-4b46-a162-e70838832041 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:53.652362 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165a371-6aa2-4b46-a162-e70838832041 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:53.652465 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165a371-6aa2-4b46-a162-e70838832041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:53.702894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5165a371-6aa2-4b46-a162-e70838832041 None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:34:53.704942 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5165a371-6aa2-4b46-a162-e70838832041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:53.775508 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1284/2578] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:34:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:34:53.796402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44aa76-b374-4ee6-b8e3-cc2831edbff8 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:53.797811 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8d44aa76-b374-4ee6-b8e3-cc2831edbff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:53.889003 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44aa76-b374-4ee6-b8e3-cc2831edbff8 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:53.894737 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a33c56-915a-4f0d-9ce5-3ea96cfaf90d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:53.895375 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a33c56-915a-4f0d-9ce5-3ea96cfaf90d service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:53.895651 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a33c56-915a-4f0d-9ce5-3ea96cfaf90d service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:53.895921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a33c56-915a-4f0d-9ce5-3ea96cfaf90d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:53.904225 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a33c56-915a-4f0d-9ce5-3ea96cfaf90d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:53.905795 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a33c56-915a-4f0d-9ce5-3ea96cfaf90d service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:53.913743 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1295/2579] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:34:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:34:54.428079 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336193f3-fe26-4e7c-a182-da8bc5e9f65e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:54.428388 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336193f3-fe26-4e7c-a182-da8bc5e9f65e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:54.428659 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336193f3-fe26-4e7c-a182-da8bc5e9f65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:54.448008 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-336193f3-fe26-4e7c-a182-da8bc5e9f65e None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:34:54.449262 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-336193f3-fe26-4e7c-a182-da8bc5e9f65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:54.456840 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1285/2580] 10.222.0.18 () {58 vars in 967 bytes} [Fri Oct 14 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:34:54.469187 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a33c56-915a-4f0d-9ce5-3ea96cfaf90d service swift] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:54.480051 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a33c56-915a-4f0d-9ce5-3ea96cfaf90d service swift] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:54.485019 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc8522-534d-43da-804b-799282dad988 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:54.485649 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc8522-534d-43da-804b-799282dad988 service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:54.485921 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc8522-534d-43da-804b-799282dad988 service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:54.486195 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc8522-534d-43da-804b-799282dad988 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:54.494103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc8522-534d-43da-804b-799282dad988 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:54.495724 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc8522-534d-43da-804b-799282dad988 service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:54.504156 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1296/2581] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:34:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:34:54.722077 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba336d-ccbd-4bc3-8efb-de322bcc51c8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:54.722562 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba336d-ccbd-4bc3-8efb-de322bcc51c8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:54.722664 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba336d-ccbd-4bc3-8efb-de322bcc51c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:54.743015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-93ba336d-ccbd-4bc3-8efb-de322bcc51c8 None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:34:54.744250 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-93ba336d-ccbd-4bc3-8efb-de322bcc51c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:54.751989 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1286/2582] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Oct 14 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:34:54.763689 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc8522-534d-43da-804b-799282dad988 service swift] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:54.774120 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc8522-534d-43da-804b-799282dad988 service swift] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:54.779170 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacea2c-d349-41b4-abb5-192bacc8deb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:54.779781 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacea2c-d349-41b4-abb5-192bacc8deb8 service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:54.780065 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacea2c-d349-41b4-abb5-192bacc8deb8 service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:54.780372 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacea2c-d349-41b4-abb5-192bacc8deb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:54.788948 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacea2c-d349-41b4-abb5-192bacc8deb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:54.790582 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacea2c-d349-41b4-abb5-192bacc8deb8 service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:54.799903 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1297/2583] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:34:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:34:55.114205 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eefcfe-bab4-448d-ae85-ea049f248692 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:55.114488 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eefcfe-bab4-448d-ae85-ea049f248692 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:55.114759 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eefcfe-bab4-448d-ae85-ea049f248692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:55.133439 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b2eefcfe-bab4-448d-ae85-ea049f248692 None None] MFA Rules not processed for user `de87b424a83f48f09fd6c89c5f5621ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:34:55.134608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b2eefcfe-bab4-448d-ae85-ea049f248692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:55.141640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1287/2584] 10.222.0.18 () {58 vars in 967 bytes} [Fri Oct 14 02:34:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:34:55.153693 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacea2c-d349-41b4-abb5-192bacc8deb8 service swift] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:55.164691 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacea2c-d349-41b4-abb5-192bacc8deb8 service swift] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:55.169723 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea470a0-fff3-4a44-b90a-edac56652233 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf0efd6d12ad458388e0c3574bbedc6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:55.170255 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea470a0-fff3-4a44-b90a-edac56652233 service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:55.170535 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea470a0-fff3-4a44-b90a-edac56652233 service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:55.170809 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea470a0-fff3-4a44-b90a-edac56652233 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:55.179340 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea470a0-fff3-4a44-b90a-edac56652233 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:55.180908 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea470a0-fff3-4a44-b90a-edac56652233 service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:55.189057 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1298/2585] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Oct 14 02:34:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 02:34:57.259698 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eefcfe-bab4-448d-ae85-ea049f248692 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:57.322242 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eefcfe-bab4-448d-ae85-ea049f248692 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:57.326796 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35bf18-68db-468a-88f2-caac03b8896e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:57.327467 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35bf18-68db-468a-88f2-caac03b8896e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:57.327745 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35bf18-68db-468a-88f2-caac03b8896e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:57.328015 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35bf18-68db-468a-88f2-caac03b8896e admin admin] PATH_INFO: `/v3/users/522a06b7de354628820de2e35ae40b63` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:57.329406 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35bf18-68db-468a-88f2-caac03b8896e admin admin] RBAC: Authorizing `identity:delete_user(user_id=522a06b7de354628820de2e35ae40b63)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:57.331391 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35bf18-68db-468a-88f2-caac03b8896e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:57.376811 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1e35bf18-68db-468a-88f2-caac03b8896e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '522a06b7de354628820de2e35ae40b63'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:57.383749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1e35bf18-68db-468a-88f2-caac03b8896e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '522a06b7de354628820de2e35ae40b63'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:57.384360 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1e35bf18-68db-468a-88f2-caac03b8896e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '522a06b7de354628820de2e35ae40b63'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:57.385485 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:34:57.399297 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1e35bf18-68db-468a-88f2-caac03b8896e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '522a06b7de354628820de2e35ae40b63'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:57.406036 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1e35bf18-68db-468a-88f2-caac03b8896e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '522a06b7de354628820de2e35ae40b63'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:57.412441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1288/2586] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:34:57 2022] DELETE /identity/v3/users/522a06b7de354628820de2e35ae40b63 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:34:57.416871 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481ab5d-00e5-4dda-83bd-ae11c0421ad2 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:57.417360 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481ab5d-00e5-4dda-83bd-ae11c0421ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:57.417469 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481ab5d-00e5-4dda-83bd-ae11c0421ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:57.436854 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c481ab5d-00e5-4dda-83bd-ae11c0421ad2 None None] MFA Rules not processed for user `6f436f18778d45bb9d581131675fcbb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:34:57.438035 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c481ab5d-00e5-4dda-83bd-ae11c0421ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:57.497977 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1299/2587] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:34:57.508060 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35bf18-68db-468a-88f2-caac03b8896e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:57.509441 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1e35bf18-68db-468a-88f2-caac03b8896e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:57.566911 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35bf18-68db-468a-88f2-caac03b8896e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:57.570640 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b1448-7025-4664-bd72-863d1cbfd099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:57.571284 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b1448-7025-4664-bd72-863d1cbfd099 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:57.571564 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b1448-7025-4664-bd72-863d1cbfd099 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:57.571836 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b1448-7025-4664-bd72-863d1cbfd099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:57.580917 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b1448-7025-4664-bd72-863d1cbfd099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:57.582333 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b1448-7025-4664-bd72-863d1cbfd099 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:57.591551 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1289/2588] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:34:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:34:57.753330 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481ab5d-00e5-4dda-83bd-ae11c0421ad2 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:34:57.754825 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c481ab5d-00e5-4dda-83bd-ae11c0421ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:34:57.767508 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481ab5d-00e5-4dda-83bd-ae11c0421ad2 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:34:57.772351 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:34:57.772986 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:34:57.773264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:34:57.773536 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929 admin admin] PATH_INFO: `/v3/projects/49bdcf3d2b67466badfe669519fe7f44` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:34:57.775060 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49bdcf3d2b67466badfe669519fe7f44)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:34:57.776578 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:34:57.808733 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:34:57.825829 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49bdcf3d2b67466badfe669519fe7f44', 'request_id': 'req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:57.832022 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49bdcf3d2b67466badfe669519fe7f44', 'request_id': 'req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:57.838250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49bdcf3d2b67466badfe669519fe7f44', 'request_id': 'req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:57.850549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929 admin admin] The token cache is being invalidate because project 49bdcf3d2b67466badfe669519fe7f44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:34:57.850835 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:57.851485 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dd71cc68-8f1f-46a4-8f47-b8e9e01f8929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:34:57.853225 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1300/2589] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:34:57 2022] DELETE /identity/v3/projects/49bdcf3d2b67466badfe669519fe7f44 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:35:07.587405 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=37,keystone:DELETE=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:35:07.847566 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:35:39.186601 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b1448-7025-4664-bd72-863d1cbfd099 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:35:39.188119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e42b1448-7025-4664-bd72-863d1cbfd099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:35:39.199269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b1448-7025-4664-bd72-863d1cbfd099 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:35:39.247192 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1f394-59ab-48c8-8ec3-9e7bbd9fbcdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:35:39.247794 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1f394-59ab-48c8-8ec3-9e7bbd9fbcdd service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:35:39.248073 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1f394-59ab-48c8-8ec3-9e7bbd9fbcdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:35:39.248345 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1f394-59ab-48c8-8ec3-9e7bbd9fbcdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:35:39.250040 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f0c1f394-59ab-48c8-8ec3-9e7bbd9fbcdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:35:39.259057 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1f394-59ab-48c8-8ec3-9e7bbd9fbcdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:35:39.260487 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1f394-59ab-48c8-8ec3-9e7bbd9fbcdd service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:35:39.269086 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1290/2590] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:35:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:35:49.264749 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=26 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:35:55.804792 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dbaf3-444c-4825-b8c6-09598a6bdb45 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:35:55.805306 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dbaf3-444c-4825-b8c6-09598a6bdb45 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:35:55.805416 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dbaf3-444c-4825-b8c6-09598a6bdb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:35:55.824549 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-cd6dbaf3-444c-4825-b8c6-09598a6bdb45 None None] MFA Rules not processed for user `eca98759b55b443b9591eb6c80501157`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:35:55.825766 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cd6dbaf3-444c-4825-b8c6-09598a6bdb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:35:55.880402 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1301/2591] 10.222.0.18 () {58 vars in 958 bytes} [Fri Oct 14 02:35:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:35:56.198353 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1f394-59ab-48c8-8ec3-9e7bbd9fbcdd service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:35:56.199773 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f0c1f394-59ab-48c8-8ec3-9e7bbd9fbcdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:35:56.210825 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1f394-59ab-48c8-8ec3-9e7bbd9fbcdd service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:35:56.214427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515193c-8213-4c17-9bff-cf8ff4f1833b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:35:56.215028 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515193c-8213-4c17-9bff-cf8ff4f1833b service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:35:56.215341 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515193c-8213-4c17-9bff-cf8ff4f1833b service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:35:56.215616 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515193c-8213-4c17-9bff-cf8ff4f1833b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:35:56.217276 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0515193c-8213-4c17-9bff-cf8ff4f1833b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:35:56.227032 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515193c-8213-4c17-9bff-cf8ff4f1833b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:35:56.228741 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515193c-8213-4c17-9bff-cf8ff4f1833b service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:35:56.238608 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1291/2592] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:35:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:35:56.855894 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6dbaf3-444c-4825-b8c6-09598a6bdb45 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:35:56.857347 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cd6dbaf3-444c-4825-b8c6-09598a6bdb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:35:56.929027 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6dbaf3-444c-4825-b8c6-09598a6bdb45 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:35:56.933973 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7889300-2551-4344-8953-57d2af93928b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e935ba63cdd4256874b99a2eb0a1a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:35:56.934583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7889300-2551-4344-8953-57d2af93928b service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:35:56.934864 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7889300-2551-4344-8953-57d2af93928b service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:35:56.935173 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7889300-2551-4344-8953-57d2af93928b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:35:56.936879 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e7889300-2551-4344-8953-57d2af93928b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:35:56.949087 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7889300-2551-4344-8953-57d2af93928b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:35:56.950672 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7889300-2551-4344-8953-57d2af93928b service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:35:56.960801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1302/2593] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:35:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:35:59.630319 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515193c-8213-4c17-9bff-cf8ff4f1833b service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:35:59.632182 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0515193c-8213-4c17-9bff-cf8ff4f1833b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:35:59.722262 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515193c-8213-4c17-9bff-cf8ff4f1833b service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:35:59.731894 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552e5f6-b270-49ef-8da9-0588a76af128 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:35:59.732563 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552e5f6-b270-49ef-8da9-0588a76af128 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:35:59.732844 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552e5f6-b270-49ef-8da9-0588a76af128 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:35:59.733119 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552e5f6-b270-49ef-8da9-0588a76af128 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:35:59.734871 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7552e5f6-b270-49ef-8da9-0588a76af128 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:35:59.748346 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552e5f6-b270-49ef-8da9-0588a76af128 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:35:59.749930 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552e5f6-b270-49ef-8da9-0588a76af128 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:35:59.761787 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1292/2594] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:35:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:36:01.800394 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7889300-2551-4344-8953-57d2af93928b service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:36:01.812450 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7889300-2551-4344-8953-57d2af93928b service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:36:01.818333 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49526da-52b7-4898-974a-0ad439c61588 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:36:01.818930 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49526da-52b7-4898-974a-0ad439c61588 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:36:01.819232 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49526da-52b7-4898-974a-0ad439c61588 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:36:01.819506 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49526da-52b7-4898-974a-0ad439c61588 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:36:01.821203 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c49526da-52b7-4898-974a-0ad439c61588 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:36:01.832608 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49526da-52b7-4898-974a-0ad439c61588 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:36:01.834183 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49526da-52b7-4898-974a-0ad439c61588 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:36:01.843209 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1303/2595] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:36:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:36:08.915765 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552e5f6-b270-49ef-8da9-0588a76af128 service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:36:08.917884 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7552e5f6-b270-49ef-8da9-0588a76af128 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:36:08.933949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552e5f6-b270-49ef-8da9-0588a76af128 service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:36:08.940210 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371294e-955a-4a70-8220-cde2da434467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:36:08.940837 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371294e-955a-4a70-8220-cde2da434467 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:36:08.941116 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371294e-955a-4a70-8220-cde2da434467 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:36:08.941387 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371294e-955a-4a70-8220-cde2da434467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:36:08.943352 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9371294e-955a-4a70-8220-cde2da434467 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:36:08.956877 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371294e-955a-4a70-8220-cde2da434467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:36:08.958526 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371294e-955a-4a70-8220-cde2da434467 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:36:08.970522 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1293/2596] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:36:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:36:11.839094 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=63 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:36:18.965422 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=46 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:37:39.525403 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49526da-52b7-4898-974a-0ad439c61588 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:37:39.526822 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c49526da-52b7-4898-974a-0ad439c61588 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:37:39.541095 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49526da-52b7-4898-974a-0ad439c61588 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:37:39.546204 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fd342-1129-4f38-a33b-fbc9c955e093 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e935ba63cdd4256874b99a2eb0a1a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:37:39.546831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fd342-1129-4f38-a33b-fbc9c955e093 service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:37:39.547141 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fd342-1129-4f38-a33b-fbc9c955e093 service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:37:39.547440 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fd342-1129-4f38-a33b-fbc9c955e093 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:37:39.549070 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e83fd342-1129-4f38-a33b-fbc9c955e093 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:37:39.561127 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fd342-1129-4f38-a33b-fbc9c955e093 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:37:39.562744 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fd342-1129-4f38-a33b-fbc9c955e093 service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:37:39.573179 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1304/2597] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:37:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:37:44.261402 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371294e-955a-4a70-8220-cde2da434467 service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:37:44.274175 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371294e-955a-4a70-8220-cde2da434467 service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:37:44.281185 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31dc2e-7dc9-40c8-8b5b-7edaea845921 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de13be55b4034477baf4c029c1a79757', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:37:44.281882 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31dc2e-7dc9-40c8-8b5b-7edaea845921 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:37:44.282379 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31dc2e-7dc9-40c8-8b5b-7edaea845921 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:37:44.282784 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31dc2e-7dc9-40c8-8b5b-7edaea845921 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:37:44.284810 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3c31dc2e-7dc9-40c8-8b5b-7edaea845921 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:37:44.299269 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31dc2e-7dc9-40c8-8b5b-7edaea845921 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:37:44.300942 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31dc2e-7dc9-40c8-8b5b-7edaea845921 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:37:44.314310 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1294/2598] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:37:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:37:49.568250 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:37:54.308496 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=6 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:37:55.342351 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fd342-1129-4f38-a33b-fbc9c955e093 service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:37:55.351605 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fd342-1129-4f38-a33b-fbc9c955e093 service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:37:55.355966 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fc21e-ec2e-4071-b9e8-dabb7d8e48e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ff451e43cea4bb0a5fd73491a05bf82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:37:55.356562 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fc21e-ec2e-4071-b9e8-dabb7d8e48e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:37:55.356843 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fc21e-ec2e-4071-b9e8-dabb7d8e48e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:37:55.357110 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fc21e-ec2e-4071-b9e8-dabb7d8e48e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:37:55.358802 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9d3fc21e-ec2e-4071-b9e8-dabb7d8e48e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:37:55.420391 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fc21e-ec2e-4071-b9e8-dabb7d8e48e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:37:55.421814 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fc21e-ec2e-4071-b9e8-dabb7d8e48e8 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:37:55.431273 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1305/2599] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:37:58.499890 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31dc2e-7dc9-40c8-8b5b-7edaea845921 service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:37:58.519735 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31dc2e-7dc9-40c8-8b5b-7edaea845921 service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:37:58.526463 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22b775-0f10-4817-9d24-17fcf1bea78f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e935ba63cdd4256874b99a2eb0a1a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:37:58.527909 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22b775-0f10-4817-9d24-17fcf1bea78f service placement] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:37:58.528435 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22b775-0f10-4817-9d24-17fcf1bea78f service placement] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:37:58.528949 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22b775-0f10-4817-9d24-17fcf1bea78f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:37:58.540726 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6a22b775-0f10-4817-9d24-17fcf1bea78f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:37:58.584679 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a22b775-0f10-4817-9d24-17fcf1bea78f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:37:58.586482 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a22b775-0f10-4817-9d24-17fcf1bea78f service placement] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:37:58.598711 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1295/2600] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:37:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:38:01.668041 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fc21e-ec2e-4071-b9e8-dabb7d8e48e8 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:38:01.679409 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fc21e-ec2e-4071-b9e8-dabb7d8e48e8 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:38:01.684983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddede891-e662-46c9-9d2b-59783371f15f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e935ba63cdd4256874b99a2eb0a1a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:38:01.685583 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddede891-e662-46c9-9d2b-59783371f15f service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:38:01.685856 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddede891-e662-46c9-9d2b-59783371f15f service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:38:01.686123 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddede891-e662-46c9-9d2b-59783371f15f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:38:01.687834 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ddede891-e662-46c9-9d2b-59783371f15f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:38:01.699624 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddede891-e662-46c9-9d2b-59783371f15f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:38:01.701213 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddede891-e662-46c9-9d2b-59783371f15f service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:38:01.711303 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1306/2601] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:38:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:38:01.741183 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22b775-0f10-4817-9d24-17fcf1bea78f service placement] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:38:01.745597 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6a22b775-0f10-4817-9d24-17fcf1bea78f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:38:01.759828 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22b775-0f10-4817-9d24-17fcf1bea78f service placement] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:38:01.764895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece1d0f-f474-4d5d-8749-753a7149dad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:38:01.765536 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece1d0f-f474-4d5d-8749-753a7149dad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:38:01.765875 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece1d0f-f474-4d5d-8749-753a7149dad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:38:01.766174 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece1d0f-f474-4d5d-8749-753a7149dad4 admin admin] PATH_INFO: `/v3/users/eca98759b55b443b9591eb6c80501157` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:38:01.767315 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece1d0f-f474-4d5d-8749-753a7149dad4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca98759b55b443b9591eb6c80501157)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:38:01.769244 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece1d0f-f474-4d5d-8749-753a7149dad4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:38:01.819427 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8ece1d0f-f474-4d5d-8749-753a7149dad4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca98759b55b443b9591eb6c80501157'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:01.828165 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8ece1d0f-f474-4d5d-8749-753a7149dad4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca98759b55b443b9591eb6c80501157'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:01.828835 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8ece1d0f-f474-4d5d-8749-753a7149dad4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca98759b55b443b9591eb6c80501157'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:01.843574 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8ece1d0f-f474-4d5d-8749-753a7149dad4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca98759b55b443b9591eb6c80501157'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:01.850369 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8ece1d0f-f474-4d5d-8749-753a7149dad4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca98759b55b443b9591eb6c80501157'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:01.856414 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1296/2602] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:38:01 2022] DELETE /identity/v3/users/eca98759b55b443b9591eb6c80501157 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:38:02.024936 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddede891-e662-46c9-9d2b-59783371f15f service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:38:02.037950 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddede891-e662-46c9-9d2b-59783371f15f service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:38:02.095983 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:38:02.096519 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:38:02.096831 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:38:02.097103 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] PATH_INFO: `/v3/projects/3d1ba99898ca42c19c8f48fa498c5851` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:38:02.098210 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d1ba99898ca42c19c8f48fa498c5851)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:38:02.100160 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:38:02.153370 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d1ba99898ca42c19c8f48fa498c5851', 'request_id': 'req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.159633 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d1ba99898ca42c19c8f48fa498c5851', 'request_id': 'req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.166106 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d1ba99898ca42c19c8f48fa498c5851', 'request_id': 'req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.179260 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] The token cache is being invalidate because project 3d1ba99898ca42c19c8f48fa498c5851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:38:02.179633 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.180264 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.181783 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1307/2603] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:38:02 2022] DELETE /identity/v3/projects/3d1ba99898ca42c19c8f48fa498c5851 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:38:02.186468 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece1d0f-f474-4d5d-8749-753a7149dad4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:38:02.187895 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8ece1d0f-f474-4d5d-8749-753a7149dad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:38:02.201625 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece1d0f-f474-4d5d-8749-753a7149dad4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:38:02.261380 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba0b75-3b26-4c94-8545-86ed54221008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:38:02.262030 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba0b75-3b26-4c94-8545-86ed54221008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:38:02.262311 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba0b75-3b26-4c94-8545-86ed54221008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:38:02.262609 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba0b75-3b26-4c94-8545-86ed54221008 admin admin] PATH_INFO: `/v3/users/127a44eadc72477a984614cc29102512` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:38:02.263748 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba0b75-3b26-4c94-8545-86ed54221008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=127a44eadc72477a984614cc29102512)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:38:02.265704 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba0b75-3b26-4c94-8545-86ed54221008 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:38:02.312154 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1aba0b75-3b26-4c94-8545-86ed54221008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127a44eadc72477a984614cc29102512'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.319129 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1aba0b75-3b26-4c94-8545-86ed54221008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127a44eadc72477a984614cc29102512'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.319738 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1aba0b75-3b26-4c94-8545-86ed54221008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127a44eadc72477a984614cc29102512'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.324991 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:38:02.333214 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1aba0b75-3b26-4c94-8545-86ed54221008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127a44eadc72477a984614cc29102512'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.340611 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1aba0b75-3b26-4c94-8545-86ed54221008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127a44eadc72477a984614cc29102512'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.346481 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1297/2604] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Oct 14 02:38:02 2022] DELETE /identity/v3/users/127a44eadc72477a984614cc29102512 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:38:02.517082 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:38:02.518592 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:38:02.532171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dd147-6bdb-43e7-be2e-c50f30c4f29e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:38:02.540801 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:38:02.591858 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f436f18778d45bb9d581131675fcbb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1e7392d098a4e219827c4ae7f8117e2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:38:02.592493 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:38:02.592776 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:38:02.593052 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8 admin admin] PATH_INFO: `/v3/projects/a3033337c7734a46a7c933fbf5325521` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:38:02.594171 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3033337c7734a46a7c933fbf5325521)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:38:02.596125 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:38:02.641567 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3033337c7734a46a7c933fbf5325521', 'request_id': 'req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.647363 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3033337c7734a46a7c933fbf5325521', 'request_id': 'req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.654214 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3033337c7734a46a7c933fbf5325521', 'request_id': 'req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.666897 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8 admin admin] The token cache is being invalidate because project a3033337c7734a46a7c933fbf5325521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 02:38:02.666897 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.667596 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a56cdf84-5933-49c9-a201-cd91f9e1b9a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 02:38:02.669046 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1308/2605] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Oct 14 02:38:02 2022] DELETE /identity/v3/projects/a3033337c7734a46a7c933fbf5325521 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:38:12.343037 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:38:12.663547 dv-cinder-845069-4-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:42:44.452888 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba0b75-3b26-4c94-8545-86ed54221008 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:42:44.454276 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1aba0b75-3b26-4c94-8545-86ed54221008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:42:44.597695 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba0b75-3b26-4c94-8545-86ed54221008 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:42:44.659783 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb76fb-197d-45c6-a925-913d92fea1f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e935ba63cdd4256874b99a2eb0a1a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d337589024cc88184355bd0ee8572', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:42:44.660399 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb76fb-197d-45c6-a925-913d92fea1f2 service placement] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:42:44.660674 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb76fb-197d-45c6-a925-913d92fea1f2 service placement] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:42:44.660945 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb76fb-197d-45c6-a925-913d92fea1f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:42:44.662681 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2deb76fb-197d-45c6-a925-913d92fea1f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:42:44.674295 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb76fb-197d-45c6-a925-913d92fea1f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:42:44.675901 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb76fb-197d-45c6-a925-913d92fea1f2 service placement] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:42:44.686685 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1298/2606] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Oct 14 02:42:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:42:54.681577 dv-cinder-845069-4-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}